Sep 15 20:02:59.901841 ip-10-0-134-155 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 15 20:02:59.901851 ip-10-0-134-155 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 15 20:02:59.901858 ip-10-0-134-155 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 15 20:02:59.901998 ip-10-0-134-155 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 15 20:03:10.151131 ip-10-0-134-155 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 15 20:03:10.151145 ip-10-0-134-155 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 8dc817d5ef6746eb85029a279ad2c5a3 --
Sep 15 20:04:44.224495 ip-10-0-134-155 systemd[1]: Starting Kubernetes Kubelet...
Sep 15 20:04:44.654664 ip-10-0-134-155 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 15 20:04:44.654664 ip-10-0-134-155 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 15 20:04:44.654664 ip-10-0-134-155 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 15 20:04:44.654664 ip-10-0-134-155 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 15 20:04:44.654664 ip-10-0-134-155 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 15 20:04:44.657076 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.656996    2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 15 20:04:44.660036 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660022    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 20:04:44.660036 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660036    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 20:04:44.660097 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660041    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 20:04:44.660097 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660044    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 20:04:44.660097 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660047    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 20:04:44.660097 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660050    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 20:04:44.660097 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660053    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 20:04:44.660097 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660055    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 20:04:44.660097 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660058    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 20:04:44.660097 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660061    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 20:04:44.660097 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660064    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 20:04:44.660097 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660066    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 20:04:44.660097 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660070    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 20:04:44.660097 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660074    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 20:04:44.660097 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660077    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 20:04:44.660097 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660083    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 20:04:44.660097 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660086    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 20:04:44.660097 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660088    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 20:04:44.660097 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660091    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 20:04:44.660097 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660094    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 20:04:44.660097 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660096    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 20:04:44.660564 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660101    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 20:04:44.660564 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660104    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 20:04:44.660564 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660107    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 20:04:44.660564 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660111    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 20:04:44.660564 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660114    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 20:04:44.660564 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660116    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 20:04:44.660564 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660119    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 20:04:44.660564 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660122    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 20:04:44.660564 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660125    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 20:04:44.660564 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660128    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 20:04:44.660564 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660130    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 20:04:44.660564 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660133    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 20:04:44.660564 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660135    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 20:04:44.660564 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660139    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 20:04:44.660564 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660141    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 20:04:44.660564 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660144    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 20:04:44.660564 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660147    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 20:04:44.660564 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660150    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 20:04:44.660564 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660152    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 20:04:44.661026 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660155    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 20:04:44.661026 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660157    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 20:04:44.661026 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660160    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 20:04:44.661026 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660163    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 20:04:44.661026 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660165    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 20:04:44.661026 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660168    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 20:04:44.661026 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660183    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 20:04:44.661026 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660186    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 20:04:44.661026 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660188    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 20:04:44.661026 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660191    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 20:04:44.661026 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660193    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 20:04:44.661026 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660196    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 20:04:44.661026 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660198    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 20:04:44.661026 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660201    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 20:04:44.661026 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660204    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 20:04:44.661026 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660207    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 20:04:44.661026 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660209    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 20:04:44.661026 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660212    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 20:04:44.661026 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660215    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 20:04:44.661542 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660217    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 20:04:44.661542 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660220    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 20:04:44.661542 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660223    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 20:04:44.661542 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660226    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 20:04:44.661542 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660229    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 20:04:44.661542 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660231    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 20:04:44.661542 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660234    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 20:04:44.661542 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660236    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 20:04:44.661542 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660239    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 20:04:44.661542 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660241    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 20:04:44.661542 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660244    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 20:04:44.661542 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660246    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 20:04:44.661542 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660249    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 20:04:44.661542 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660252    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 20:04:44.661542 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660254    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 20:04:44.661542 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660257    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 20:04:44.661542 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660259    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 20:04:44.661542 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660262    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 20:04:44.661542 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660264    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 20:04:44.662001 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660266    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 20:04:44.662001 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660269    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 20:04:44.662001 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660271    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 20:04:44.662001 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660274    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 20:04:44.662001 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660277    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 20:04:44.662001 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660279    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 20:04:44.662001 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660282    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 20:04:44.662001 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660285    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 20:04:44.662001 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660287    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 20:04:44.662001 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660290    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 20:04:44.662001 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660293    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 20:04:44.662001 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660295    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 20:04:44.662001 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660298    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 20:04:44.662001 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660300    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 20:04:44.662001 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660303    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 20:04:44.662001 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660305    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 20:04:44.662001 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660309    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 20:04:44.662001 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660313    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 20:04:44.662001 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660316    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 20:04:44.662001 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660318    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 20:04:44.662507 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660321    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 20:04:44.662507 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660952    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 20:04:44.662507 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660958    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 20:04:44.662507 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660961    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 20:04:44.662507 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660966    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 20:04:44.662507 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660969    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 20:04:44.662507 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660973    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 20:04:44.662507 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660976    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 20:04:44.662507 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660979    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 20:04:44.662507 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660982    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 20:04:44.662507 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660984    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 20:04:44.662507 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660987    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 20:04:44.662507 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660989    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 20:04:44.662507 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660992    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 20:04:44.662507 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660995    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 20:04:44.662507 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.660997    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 20:04:44.662507 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661000    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 20:04:44.662507 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661003    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 20:04:44.662507 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661005    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 20:04:44.662507 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661008    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 20:04:44.663010 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661011    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 20:04:44.663010 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661013    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 20:04:44.663010 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661016    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 20:04:44.663010 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661020    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 20:04:44.663010 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661024    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 20:04:44.663010 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661027    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 20:04:44.663010 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661030    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 20:04:44.663010 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661033    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 20:04:44.663010 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661037    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 20:04:44.663010 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661039    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 20:04:44.663010 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661043    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 20:04:44.663010 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661045    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 20:04:44.663010 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661048    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 20:04:44.663010 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661050    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 20:04:44.663010 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661053    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 20:04:44.663010 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661055    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 20:04:44.663010 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661058    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 20:04:44.663010 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661061    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 20:04:44.663010 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661063    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 20:04:44.663487 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661066    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 20:04:44.663487 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661069    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 20:04:44.663487 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661072    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 20:04:44.663487 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661075    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 20:04:44.663487 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661077    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 20:04:44.663487 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661080    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 20:04:44.663487 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661082    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 20:04:44.663487 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661085    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 20:04:44.663487 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661089    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 20:04:44.663487 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661091    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 20:04:44.663487 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661094    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 20:04:44.663487 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661096    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 20:04:44.663487 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661099    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 20:04:44.663487 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661102    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 20:04:44.663487 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661104    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 20:04:44.663487 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661106    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 20:04:44.663487 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661109    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 20:04:44.663487 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661112    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 20:04:44.663487 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661114    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 20:04:44.663487 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661117    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 20:04:44.663981 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661119    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 20:04:44.663981 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661122    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 20:04:44.663981 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661124    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 20:04:44.663981 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661127    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 20:04:44.663981 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661129    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 20:04:44.663981 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661132    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 20:04:44.663981 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661134    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 20:04:44.663981 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661137    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 20:04:44.663981 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661139    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 20:04:44.663981 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661143    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 20:04:44.663981 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661146    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 20:04:44.663981 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661148    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 20:04:44.663981 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661151    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 20:04:44.663981 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661154    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 20:04:44.663981 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661157    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 20:04:44.663981 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661159    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 20:04:44.663981 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661163    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 20:04:44.663981 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661166    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 20:04:44.663981 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661169    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 20:04:44.664472 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661185    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 20:04:44.664472 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661189    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 20:04:44.664472 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661192    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 20:04:44.664472 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661195    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 20:04:44.664472 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661198    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 20:04:44.664472 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661200    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 20:04:44.664472 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661203    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 20:04:44.664472 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661206    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 20:04:44.664472 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661208    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 20:04:44.664472 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661211    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 20:04:44.664472 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661213    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 20:04:44.664472 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661216    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 20:04:44.664472 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661219    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 20:04:44.664472 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661221    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 20:04:44.664472 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661223    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 20:04:44.664472 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661226    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 20:04:44.664472 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661229    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 20:04:44.664472 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661231    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 20:04:44.664472 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661234    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 20:04:44.664472 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661236    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 20:04:44.664961 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661239    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 20:04:44.664961 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661241    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 20:04:44.664961 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661652    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 20:04:44.664961 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661658    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 20:04:44.664961 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661661    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 20:04:44.664961 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661664    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 20:04:44.664961 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661668    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 20:04:44.664961 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661671    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 20:04:44.664961 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661674    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 20:04:44.664961 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661676    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 20:04:44.664961 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661679    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 20:04:44.664961 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661681    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 20:04:44.664961 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661684    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 20:04:44.664961 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661686    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 20:04:44.664961 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661689    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 20:04:44.664961 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661692    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 20:04:44.664961 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661695    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 20:04:44.664961 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661697    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 20:04:44.664961 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661699    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 20:04:44.664961 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661702    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 20:04:44.665476 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661704    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 20:04:44.665476 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661707    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 20:04:44.665476 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661709    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 20:04:44.665476 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661712    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 20:04:44.665476 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661714    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 20:04:44.665476 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661717    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 20:04:44.665476 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661720    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 20:04:44.665476 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661723    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 20:04:44.665476 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661725    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 20:04:44.665476 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661727    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 20:04:44.665476 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661730    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 20:04:44.665476 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661733    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 20:04:44.665476 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661735    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 20:04:44.665476 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661738    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 20:04:44.665476 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661741    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 20:04:44.665476 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661743    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 20:04:44.665476 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661746    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 20:04:44.665476 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661750    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 20:04:44.665476 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661752    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 20:04:44.665927 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661755    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 20:04:44.665927 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661758    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 20:04:44.665927 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661760    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 20:04:44.665927 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661763    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 20:04:44.665927 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661767    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 20:04:44.665927 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661771    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 20:04:44.665927 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661774    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 20:04:44.665927 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661776    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 20:04:44.665927 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661779    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 20:04:44.665927 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661781    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 20:04:44.665927 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661784    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 20:04:44.665927 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661786    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 20:04:44.665927 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661789    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 20:04:44.665927 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661791    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 20:04:44.665927 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661794    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 20:04:44.665927 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661797    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 20:04:44.665927 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661799    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 20:04:44.665927 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661803    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 20:04:44.665927 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661806    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 20:04:44.665927 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661809    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 20:04:44.666480 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661812    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 20:04:44.666480 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661815    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 20:04:44.666480 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661818    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 20:04:44.666480 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661820    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 20:04:44.666480 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661823    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 20:04:44.666480 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661826    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 20:04:44.666480 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661829    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 20:04:44.666480 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661831    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 20:04:44.666480 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661834    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 20:04:44.666480 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661836    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 20:04:44.666480 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661839    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 20:04:44.666480 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661841    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 20:04:44.666480 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661844    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 20:04:44.666480 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661847    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 20:04:44.666480 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661849    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 20:04:44.666480 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661852    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 20:04:44.666480 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661854    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 20:04:44.666480 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661857    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 20:04:44.666480 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661859    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 20:04:44.666480 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661862    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 20:04:44.666963 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661865    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 20:04:44.666963 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661867    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 20:04:44.666963 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661870    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 20:04:44.666963 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661872    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 20:04:44.666963 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661874    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 20:04:44.666963 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661877    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 20:04:44.666963 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661879    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 20:04:44.666963 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661882    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 20:04:44.666963 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661884    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 20:04:44.666963 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661887    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 20:04:44.666963 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661890    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 20:04:44.666963 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661893    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 20:04:44.666963 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661895    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 20:04:44.666963 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661898    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 20:04:44.666963 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661901    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 20:04:44.666963 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661905    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 20:04:44.666963 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661908    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 20:04:44.666963 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661910    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 20:04:44.666963 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661913    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 20:04:44.667457 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661916    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 20:04:44.667457 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661919    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 20:04:44.667457 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.661922    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 20:04:44.667457 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.661989    2576 flags.go:64] FLAG: --address="0.0.0.0"
Sep 15 20:04:44.667457 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662027    2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 15 20:04:44.667457 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662035    2576 flags.go:64] FLAG: --anonymous-auth="true"
Sep 15 20:04:44.667457 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662040    2576 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 15 20:04:44.667457 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662045    2576 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 15 20:04:44.667457 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662049    2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 15 20:04:44.667457 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662053    2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 15 20:04:44.667457 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662058    2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 15 20:04:44.667457 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662061    2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 15 20:04:44.667457 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662064    2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 15 20:04:44.667457 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662067    2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 15 20:04:44.667457 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662070    2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 15 20:04:44.667457 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662074    2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 15 20:04:44.667457 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662077    2576 flags.go:64] FLAG: --cgroup-root=""
Sep 15 20:04:44.667457 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662080    2576 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 15 20:04:44.667457 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662083    2576 flags.go:64] FLAG: --client-ca-file=""
Sep 15 20:04:44.667457 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662086    2576 flags.go:64] FLAG: --cloud-provider="external"
Sep 15 20:04:44.667457 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662088    2576 flags.go:64] FLAG: --cluster-dns="[]"
Sep 15 20:04:44.667457 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662093    2576 flags.go:64] FLAG: --cluster-domain=""
Sep 15 20:04:44.667457 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662096    2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 15 20:04:44.667457 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662099    2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 15 20:04:44.668040 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662102    2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 15 20:04:44.668040 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662106    2576 flags.go:64] FLAG: --container-log-max-files="5"
Sep 15 20:04:44.668040 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662109    2576 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 15 20:04:44.668040 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662112    2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 15 20:04:44.668040 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662115    2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 15 20:04:44.668040 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662118    2576 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 15 20:04:44.668040 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662121    2576 flags.go:64] FLAG: --contention-profiling="false"
Sep 15 20:04:44.668040 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662124    2576 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 15 20:04:44.668040 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662127    2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 15 20:04:44.668040 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662131    2576 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 15 20:04:44.668040 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662134    2576 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 15 20:04:44.668040 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662140    2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 15 20:04:44.668040 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662143    2576 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 15 20:04:44.668040 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662146    2576 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 15 20:04:44.668040 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662149    2576 flags.go:64] FLAG: --enable-load-reader="false"
Sep 15 20:04:44.668040 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662152    2576 flags.go:64] FLAG: --enable-server="true"
Sep 15 20:04:44.668040 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662156    2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 15 20:04:44.668040 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662160    2576 flags.go:64] FLAG: --event-burst="100"
Sep 15 20:04:44.668040 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662163    2576 flags.go:64] FLAG: --event-qps="50"
Sep 15 20:04:44.668040 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662166    2576 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 15 20:04:44.668040 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662169    2576 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 15 20:04:44.668040 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662184    2576 flags.go:64] FLAG: --eviction-hard=""
Sep 15 20:04:44.668040 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662188    2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 15 20:04:44.668040 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662191    2576 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 15 20:04:44.668628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662194    2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 15 20:04:44.668628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662197    2576 flags.go:64] FLAG: --eviction-soft=""
Sep 15 20:04:44.668628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662200    2576 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 15 20:04:44.668628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662203    2576 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 15 20:04:44.668628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662210    2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 15 20:04:44.668628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662213    2576 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 15 20:04:44.668628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662216    2576 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 15 20:04:44.668628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662219    2576 flags.go:64] FLAG: --fail-swap-on="true"
Sep 15 20:04:44.668628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662222    2576 flags.go:64] FLAG: --feature-gates=""
Sep 15 20:04:44.668628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662226    2576 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 15 20:04:44.668628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662229    2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 15 20:04:44.668628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662233    2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 15 20:04:44.668628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662236    2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 15 20:04:44.668628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662239    2576 flags.go:64] FLAG: --healthz-port="10248"
Sep 15 20:04:44.668628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662242    2576 flags.go:64] FLAG: --help="false"
Sep 15 20:04:44.668628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662245    2576 flags.go:64] FLAG: --hostname-override="ip-10-0-134-155.ec2.internal"
Sep 15 20:04:44.668628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662248    2576 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 15 20:04:44.668628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662251    2576 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 15 20:04:44.668628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662254    2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 15 20:04:44.668628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662259    2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 15 20:04:44.668628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662262    2576 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 15 20:04:44.668628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662265    2576 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 15 20:04:44.668628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662268    2576 flags.go:64] FLAG: --image-service-endpoint=""
Sep 15 20:04:44.669253 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662271    2576 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 15 20:04:44.669253 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662273    2576 flags.go:64] FLAG: --kube-api-burst="100"
Sep 15 20:04:44.669253 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662277    2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 15 20:04:44.669253 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662280    2576 flags.go:64] FLAG: --kube-api-qps="50"
Sep 15 20:04:44.669253 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662283    2576 flags.go:64] FLAG: --kube-reserved=""
Sep 15 20:04:44.669253 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662286    2576 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 15 20:04:44.669253 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662289    2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 15 20:04:44.669253 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662292    2576 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 15 20:04:44.669253 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662294    2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 15 20:04:44.669253 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662297    2576 flags.go:64] FLAG: --lock-file=""
Sep 15 20:04:44.669253 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662300    2576 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 15 20:04:44.669253 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662303    2576 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 15 20:04:44.669253 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662306    2576 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 15 20:04:44.669253 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662313    2576 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 15 20:04:44.669253 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662316    2576 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 15 20:04:44.669253 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662319    2576 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 15 20:04:44.669253 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662321    2576 flags.go:64] FLAG: --logging-format="text"
Sep 15 20:04:44.669253 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662324    2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 15 20:04:44.669253 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662327    2576 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 15 20:04:44.669253 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662330    2576 flags.go:64] FLAG: --manifest-url=""
Sep 15 20:04:44.669253 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662334    2576 flags.go:64] FLAG: --manifest-url-header=""
Sep 15 20:04:44.669253 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662338    2576 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 15 20:04:44.669253 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662341    2576 flags.go:64] FLAG: --max-open-files="1000000"
Sep 15 20:04:44.669253 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662344    2576 flags.go:64] FLAG: --max-pods="110"
Sep 15 20:04:44.669253 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662347    2576 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 15 20:04:44.669894 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662350    2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 15 20:04:44.669894 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662353    2576 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 15 20:04:44.669894 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662356    2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 15 20:04:44.669894 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662360    2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 15 20:04:44.669894 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662363    2576 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 15 20:04:44.669894 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662366    2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 15 20:04:44.669894 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662373    2576 flags.go:64] FLAG: --node-status-max-images="50"
Sep 15 20:04:44.669894 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662383    2576 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 15 20:04:44.669894 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662386    2576 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 15 20:04:44.669894 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662389    2576 flags.go:64] FLAG: --pod-cidr=""
Sep 15 20:04:44.669894 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662392    2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 15 20:04:44.669894 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662397    2576 flags.go:64] FLAG: --pod-manifest-path=""
Sep 15 20:04:44.669894 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662400    2576 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 15 20:04:44.669894 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662403    2576 flags.go:64] FLAG: --pods-per-core="0"
Sep 15 20:04:44.669894 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662406    2576 flags.go:64] FLAG: --port="10250"
Sep 15 20:04:44.669894 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662409    2576 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 15 20:04:44.669894 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662412    2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f77ea9d16f9e6974"
Sep 15 20:04:44.669894 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662416    2576 flags.go:64] FLAG: --qos-reserved=""
Sep 15 20:04:44.669894 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662418    2576 flags.go:64] FLAG: --read-only-port="10255"
Sep 15 20:04:44.669894 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662421    2576 flags.go:64] FLAG: --register-node="true"
Sep 15 20:04:44.669894 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662426    2576 flags.go:64] FLAG: --register-with-taints=""
Sep 15 20:04:44.669894 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662429    2576 flags.go:64] FLAG: --registry-burst="10"
Sep 15 20:04:44.669894 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662432    2576 flags.go:64] FLAG: --registry-qps="5"
Sep 15 20:04:44.669894 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662435    2576 flags.go:64] FLAG: --reserved-cpus=""
Sep 15 20:04:44.670491 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662438    2576 flags.go:64] FLAG: --reserved-memory=""
Sep 15 20:04:44.670491 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662441    2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 15 20:04:44.670491 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662444    2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 15 20:04:44.670491 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662447    2576 flags.go:64] FLAG: --rotate-certificates="false"
Sep 15 20:04:44.670491 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662450    2576 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 15 20:04:44.670491 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662453    2576 flags.go:64] FLAG: --runonce="false"
Sep 15 20:04:44.670491 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662456    2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 15 20:04:44.670491 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662459    2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 15 20:04:44.670491 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662462    2576 flags.go:64] FLAG: --seccomp-default="false"
Sep 15 20:04:44.670491 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662465    2576 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 15 20:04:44.670491 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662468    2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 15 20:04:44.670491 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662471    2576 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 15 20:04:44.670491 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662476    2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 15 20:04:44.670491 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662479    2576 flags.go:64] FLAG: --storage-driver-password="root"
Sep 15 20:04:44.670491 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662482    2576 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 15 20:04:44.670491 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662484    2576 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 15 20:04:44.670491 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662487    2576 flags.go:64] FLAG: --storage-driver-user="root"
Sep 15 20:04:44.670491 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662490    2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 15 20:04:44.670491 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662493    2576 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 15 20:04:44.670491 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662497    2576 flags.go:64] FLAG: --system-cgroups=""
Sep 15 20:04:44.670491 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662500    2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 15 20:04:44.670491 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662505    2576 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 15 20:04:44.670491 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662508    2576 flags.go:64] FLAG: --tls-cert-file=""
Sep 15 20:04:44.670491 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662511    2576 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 15 20:04:44.670491 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662516    2576 flags.go:64] FLAG: --tls-min-version=""
Sep 15 20:04:44.671105 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662519    2576 flags.go:64] FLAG: --tls-private-key-file=""
Sep 15 20:04:44.671105 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662522    2576 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 15 20:04:44.671105 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662525    2576 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 15 20:04:44.671105 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662534    2576 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 15 20:04:44.671105 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662537    2576 flags.go:64] FLAG: --v="2"
Sep 15 20:04:44.671105 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662541    2576 flags.go:64] FLAG: --version="false"
Sep 15 20:04:44.671105 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662545    2576 flags.go:64] FLAG: --vmodule=""
Sep 15 20:04:44.671105 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662549    2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 15 20:04:44.671105 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.662552    2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 15 20:04:44.671105 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662659    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 20:04:44.671105 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662663    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 20:04:44.671105 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662666    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 20:04:44.671105 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662669    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 20:04:44.671105 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662671    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 20:04:44.671105 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662674    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 20:04:44.671105 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662677    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 20:04:44.671105 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662680    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 20:04:44.671105 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662682    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 20:04:44.671105 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662685    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 20:04:44.671105 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662689    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 20:04:44.671105 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662692    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 20:04:44.671105 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662695    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 20:04:44.671672 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662697    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 20:04:44.671672 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662700    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 20:04:44.671672 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662703    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 20:04:44.671672 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662706    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 20:04:44.671672 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662708    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 20:04:44.671672 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662711    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 20:04:44.671672 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662716    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 20:04:44.671672 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662719    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 20:04:44.671672 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662722    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 20:04:44.671672 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662725    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 20:04:44.671672 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662728    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 20:04:44.671672 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662731    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 20:04:44.671672 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662734    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 20:04:44.671672 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662737    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 20:04:44.671672 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662740    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 20:04:44.671672 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662743    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 20:04:44.671672 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662746    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 20:04:44.671672 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662749    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 20:04:44.671672 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662751    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 20:04:44.672133 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662754    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 20:04:44.672133 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662757    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 20:04:44.672133 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662759    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 20:04:44.672133 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662762    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 20:04:44.672133 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662765    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 20:04:44.672133 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662768    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 20:04:44.672133 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662770    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 20:04:44.672133 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662773    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 20:04:44.672133 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662776    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 20:04:44.672133 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662778    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 20:04:44.672133 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662782    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 20:04:44.672133 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662785    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 20:04:44.672133 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662787    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 20:04:44.672133 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662790    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 20:04:44.672133 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662793    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 20:04:44.672133 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662795    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 20:04:44.672133 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662798    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 20:04:44.672133 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662801    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 20:04:44.672133 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662803    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 20:04:44.672133 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662807    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 20:04:44.672656 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662809    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 20:04:44.672656 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662812    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 20:04:44.672656 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662815    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 20:04:44.672656 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662817    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 20:04:44.672656 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662820    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 20:04:44.672656 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662823    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 20:04:44.672656 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662827    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 20:04:44.672656 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662829    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 20:04:44.672656 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662833    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 20:04:44.672656 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662837    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 20:04:44.672656 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662839    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 20:04:44.672656 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662842    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 20:04:44.672656 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662845    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 20:04:44.672656 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662847    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 20:04:44.672656 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662850    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 20:04:44.672656 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662853    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 20:04:44.672656 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662855    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 20:04:44.672656 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662858    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 20:04:44.672656 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662861    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 20:04:44.673285 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662863    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 20:04:44.673285 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662866    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 20:04:44.673285 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662869    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 20:04:44.673285 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662873    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 20:04:44.673285 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662875    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 20:04:44.673285 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662878    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 20:04:44.673285 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662881    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 20:04:44.673285 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662883    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 20:04:44.673285 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662886    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 20:04:44.673285 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662890    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 20:04:44.673285 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662893    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 20:04:44.673285 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662897    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 20:04:44.673285 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662900    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 20:04:44.673285 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662903    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 20:04:44.673285 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662906    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 20:04:44.673285 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662909    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 20:04:44.673285 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662911    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 20:04:44.673285 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662914    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 20:04:44.673285 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662916    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 20:04:44.673745 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662920    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 20:04:44.673745 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662923    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 20:04:44.673745 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662926    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 20:04:44.673745 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662928    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 20:04:44.673745 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662931    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 20:04:44.673745 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662934    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 20:04:44.673745 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662937    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 20:04:44.673745 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662939    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 20:04:44.673745 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.662942    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 20:04:44.673745 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.663598    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 15 20:04:44.673745 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.670566    2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 15 20:04:44.673745 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.670581    2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 15 20:04:44.673745 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.671430    2576 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 15 20:04:44.673745 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.671437    2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 15 20:04:44.673745 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671494    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 20:04:44.674116 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671498    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 20:04:44.674116 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671502    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 20:04:44.674116 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671504    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 20:04:44.674116 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671507    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 20:04:44.674116 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671510    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 20:04:44.674116 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671513    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 20:04:44.674116 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671515    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 20:04:44.674116 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671519    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 20:04:44.674116 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671523    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 20:04:44.674116 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671526    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 20:04:44.674116 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671529    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 20:04:44.674116 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671533    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 20:04:44.674116 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671536    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 20:04:44.674116 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671539    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 20:04:44.674116 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671542    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 20:04:44.674116 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671545    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 20:04:44.674116 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671547    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 20:04:44.674116 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671550    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 20:04:44.674116 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671553    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 20:04:44.674116 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671555    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 20:04:44.674643 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671558    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 20:04:44.674643 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671560    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 20:04:44.674643 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671563    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 20:04:44.674643 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671566    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 20:04:44.674643 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671568    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 20:04:44.674643 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671570    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 20:04:44.674643 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671573    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 20:04:44.674643 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671575    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 20:04:44.674643 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671579    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 20:04:44.674643 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671583    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 20:04:44.674643 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671587    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 20:04:44.674643 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671590    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 20:04:44.674643 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671593    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 20:04:44.674643 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671596    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 20:04:44.674643 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671599    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 20:04:44.674643 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671602    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 20:04:44.674643 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671604    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 20:04:44.674643 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671607    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 20:04:44.674643 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671610    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 20:04:44.675115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671612    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 20:04:44.675115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671615    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 20:04:44.675115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671617    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 20:04:44.675115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671619    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 20:04:44.675115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671622    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 20:04:44.675115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671625    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 20:04:44.675115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671628    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 20:04:44.675115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671631    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 20:04:44.675115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671633    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 20:04:44.675115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671636    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 20:04:44.675115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671638    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 20:04:44.675115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671641    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 20:04:44.675115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671643    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 20:04:44.675115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671646    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 20:04:44.675115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671648    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 20:04:44.675115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671651    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 20:04:44.675115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671653    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 20:04:44.675115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671656    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 20:04:44.675115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671666    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 20:04:44.675115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671669    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 20:04:44.675644 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671673    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 20:04:44.675644 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671676    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 20:04:44.675644 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671679    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 20:04:44.675644 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671682    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 20:04:44.675644 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671685    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 20:04:44.675644 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671688    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 20:04:44.675644 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671691    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 20:04:44.675644 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671694    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 20:04:44.675644 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671697    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 20:04:44.675644 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671700    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 20:04:44.675644 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671702    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 20:04:44.675644 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671705    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 20:04:44.675644 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671707    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 20:04:44.675644 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671710    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 20:04:44.675644 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671713    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 20:04:44.675644 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671715    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 20:04:44.675644 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671719    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 20:04:44.675644 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671722    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 20:04:44.675644 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671726    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 20:04:44.676115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671728    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 20:04:44.676115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671731    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 20:04:44.676115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671733    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 20:04:44.676115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671736    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 20:04:44.676115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671738    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 20:04:44.676115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671741    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 20:04:44.676115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671743    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 20:04:44.676115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671746    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 20:04:44.676115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671748    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 20:04:44.676115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671751    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 20:04:44.676115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671753    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 20:04:44.676115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671756    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 20:04:44.676115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671758    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 20:04:44.676115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671761    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 20:04:44.676115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671764    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 20:04:44.676115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671767    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 20:04:44.676115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671770    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 20:04:44.676115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671773    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 20:04:44.676115 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671775    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 20:04:44.676687 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671778    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 20:04:44.676687 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.671783    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 15 20:04:44.676687 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671890    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 20:04:44.676687 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671894    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 20:04:44.676687 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671897    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 20:04:44.676687 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671900    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 20:04:44.676687 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671903    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 20:04:44.676687 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671906    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 20:04:44.676687 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671908    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 20:04:44.676687 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671911    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 20:04:44.676687 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671914    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 20:04:44.676687 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671916    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 20:04:44.676687 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671919    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 20:04:44.676687 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671922    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 20:04:44.676687 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671925    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 20:04:44.677061 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671927    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 20:04:44.677061 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671930    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 20:04:44.677061 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671932    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 20:04:44.677061 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671936    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 20:04:44.677061 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671939    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 20:04:44.677061 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671942    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 20:04:44.677061 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671944    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 20:04:44.677061 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671947    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 20:04:44.677061 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671950    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 20:04:44.677061 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671952    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 20:04:44.677061 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671955    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 20:04:44.677061 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671958    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 20:04:44.677061 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671961    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 20:04:44.677061 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671964    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 20:04:44.677061 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671967    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 20:04:44.677061 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671971    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 20:04:44.677061 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671974    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 20:04:44.677061 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671977    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 20:04:44.677509 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671980    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 20:04:44.677509 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671983    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 20:04:44.677509 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671986    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 20:04:44.677509 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671988    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 20:04:44.677509 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671991    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 20:04:44.677509 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671993    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 20:04:44.677509 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671996    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 20:04:44.677509 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.671999    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 20:04:44.677509 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672001    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 20:04:44.677509 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672004    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 20:04:44.677509 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672006    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 20:04:44.677509 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672009    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 20:04:44.677509 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672011    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 20:04:44.677509 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672014    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 20:04:44.677509 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672017    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 20:04:44.677509 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672019    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 20:04:44.677509 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672022    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 20:04:44.677509 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672025    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 20:04:44.677509 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672027    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 20:04:44.677509 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672030    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 20:04:44.677994 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672032    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 20:04:44.677994 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672035    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 20:04:44.677994 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672039    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 20:04:44.677994 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672042    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 20:04:44.677994 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672045    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 20:04:44.677994 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672048    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 20:04:44.677994 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672052    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 20:04:44.677994 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672055    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 20:04:44.677994 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672058    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 20:04:44.677994 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672060    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 20:04:44.677994 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672063    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 20:04:44.677994 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672065    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 20:04:44.677994 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672068    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 20:04:44.677994 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672070    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 20:04:44.677994 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672073    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 20:04:44.677994 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672075    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 20:04:44.677994 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672078    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 20:04:44.677994 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672080    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 20:04:44.677994 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672083    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 20:04:44.678479 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672085    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 20:04:44.678479 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672088    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 20:04:44.678479 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672090    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 20:04:44.678479 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672093    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 20:04:44.678479 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672095    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 20:04:44.678479 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672098    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 20:04:44.678479 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672100    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 20:04:44.678479 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672103    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 20:04:44.678479 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672105    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 20:04:44.678479 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672108    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 20:04:44.678479 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672110    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 20:04:44.678479 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672113    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 20:04:44.678479 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672115    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 20:04:44.678479 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672118    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 20:04:44.678479 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672120    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 20:04:44.678479 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672123    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 20:04:44.678479 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672125    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 20:04:44.678479 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672128    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 20:04:44.678479 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672131    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 20:04:44.678479 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672133    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 20:04:44.679006 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672136    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 20:04:44.679006 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672139    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 20:04:44.679006 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672142    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 20:04:44.679006 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672144    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 20:04:44.679006 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672147    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 20:04:44.679006 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672150    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 20:04:44.679006 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672152    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 20:04:44.679006 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672155    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 20:04:44.679006 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:44.672157    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 20:04:44.679006 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.672162    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 15 20:04:44.679006 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.672267    2576 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 15 20:04:44.679006 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.674425    2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 15 20:04:44.679006 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.675496    2576 server.go:1019] "Starting client certificate rotation"
Sep 15 20:04:44.679006 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.675599    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 15 20:04:44.679373 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.675640    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 15 20:04:44.679373 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.678825    2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 15 20:04:44.683366 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.683343    2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 15 20:04:44.691076 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.691057    2576 log.go:25] "Validated CRI v1 runtime API"
Sep 15 20:04:44.694069 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.694054    2576 log.go:25] "Validated CRI v1 image API"
Sep 15 20:04:44.694910 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.694894    2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 15 20:04:44.699301 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.699276    2576 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 857740b3-6022-4ae3-a7fd-850cad762810:/dev/nvme0n1p4 aef3cada-3043-406d-b9e9-108805f2ef66:/dev/nvme0n1p3]
Sep 15 20:04:44.699347 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.699300    2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 15 20:04:44.704530 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.704428    2576 manager.go:216] Machine: {Timestamp:2026-09-15 20:04:44.702920796 +0000 UTC m=+0.363131923 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102251 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d418b8cbb1740dd67777c9ff6be5d SystemUUID:ec2d418b-8cbb-1740-dd67-777c9ff6be5d BootID:8dc817d5-ef67-46eb-8502-9a279ad2c5a3 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:cc:18:29:bf Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:cc:18:29:bf Speed:0 Mtu:9001} {Name:ovs-system MacAddress:4e:56:bd:5a:73:76 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 15 20:04:44.704530 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.704528    2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 15 20:04:44.704638 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.704627    2576 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 15 20:04:44.707909 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.707883    2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 15 20:04:44.708040 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.707911    2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-155.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 15 20:04:44.708704 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.708694    2576 topology_manager.go:138] "Creating topology manager with none policy"
Sep 15 20:04:44.708734 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.708707    2576 container_manager_linux.go:309] "Creating device plugin manager"
Sep 15 20:04:44.708734 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.708713    2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 15 20:04:44.708734 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.708727    2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 15 20:04:44.709786 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.709775    2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 15 20:04:44.712051 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.712036    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 15 20:04:44.714141 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.714129    2576 state_mem.go:36] "Initialized new in-memory state store"
Sep 15 20:04:44.714253 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.714243    2576 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 15 20:04:44.714289 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.714270    2576 kubelet.go:486] "Attempting to sync node with API server"
Sep 15 20:04:44.714289 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.714280    2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 15 20:04:44.714345 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.714295    2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 15 20:04:44.714345 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.714304    2576 kubelet.go:398] "Adding apiserver pod source"
Sep 15 20:04:44.714345 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.714312    2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 15 20:04:44.714725 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.714713    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 15 20:04:44.714760 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.714733    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 15 20:04:44.717843 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.717828    2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 15 20:04:44.720800 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.720697    2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 15 20:04:44.720899 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.720868    2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 15 20:04:44.723279 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.723263    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 15 20:04:44.723279 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.723280    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 15 20:04:44.723393 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.723286    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 15 20:04:44.723393 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.723293    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 15 20:04:44.723393 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.723302    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 15 20:04:44.723393 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.723308    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 15 20:04:44.723393 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.723323    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 15 20:04:44.723393 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.723333    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 15 20:04:44.723393 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.723347    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 15 20:04:44.723393 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.723360    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 15 20:04:44.723393 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.723369    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 15 20:04:44.723393 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.723374    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 15 20:04:44.724506 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.724482    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 15 20:04:44.724596 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.724512    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 15 20:04:44.725902 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:44.725875    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 15 20:04:44.725902 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:44.725864    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-155.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 15 20:04:44.728318 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.728304    2576 server.go:1268] "Started kubelet"
Sep 15 20:04:44.728426 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.728380    2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 15 20:04:44.728487 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.728390    2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 15 20:04:44.728487 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.728448    2576 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 15 20:04:44.729040 ip-10-0-134-155 systemd[1]: Started Kubernetes Kubelet.
Sep 15 20:04:44.730139 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.730124    2576 server.go:310] "Adding debug handlers to kubelet server"
Sep 15 20:04:44.730419 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.730402    2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 15 20:04:44.732511 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:44.732492    2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 15 20:04:44.737911 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.737896    2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-155.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 15 20:04:44.738797 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:44.737668    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-155.ec2.internal.18d59735e7d6328c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-155.ec2.internal,UID:ip-10-0-134-155.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-155.ec2.internal,},FirstTimestamp:2026-09-15 20:04:44.728275596 +0000 UTC m=+0.388486723,LastTimestamp:2026-09-15 20:04:44.728275596 +0000 UTC m=+0.388486723,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-155.ec2.internal,}"
Sep 15 20:04:44.740340 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.740325    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 15 20:04:44.741027 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.741008    2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 15 20:04:44.741210 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:44.741191    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-155.ec2.internal\" not found"
Sep 15 20:04:44.741785 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.741768    2576 factory.go:55] Registering systemd factory
Sep 15 20:04:44.741869 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.741788    2576 factory.go:223] Registration of the systemd container factory successfully
Sep 15 20:04:44.741869 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.741832    2576 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 15 20:04:44.741869 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.741847    2576 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 15 20:04:44.741869 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.741831    2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 15 20:04:44.742047 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.741911    2576 reconstruct.go:97] "Volume reconstruction finished"
Sep 15 20:04:44.742047 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.741917    2576 reconciler.go:29] "Reconciler: start to sync state"
Sep 15 20:04:44.742047 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.742020    2576 factory.go:153] Registering CRI-O factory
Sep 15 20:04:44.742047 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.742032    2576 factory.go:223] Registration of the crio container factory successfully
Sep 15 20:04:44.742251 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.742091    2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 15 20:04:44.742251 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.742107    2576 factory.go:103] Registering Raw factory
Sep 15 20:04:44.742251 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.742127    2576 manager.go:1196] Started watching for new ooms in manager
Sep 15 20:04:44.742510 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.742498    2576 manager.go:349] Starting recovery of all containers
Sep 15 20:04:44.747001 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:44.746975    2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-155.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 15 20:04:44.747082 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:44.747032    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 15 20:04:44.754202 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.754168    2576 manager.go:354] Recovery completed
Sep 15 20:04:44.760016 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.760003    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 20:04:44.761589 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.761571    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-155.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 20:04:44.761648 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.761602    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-155.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 20:04:44.761648 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.761616    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-155.ec2.internal" event="NodeHasSufficientPID"
Sep 15 20:04:44.763073 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.762298    2576 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 15 20:04:44.763073 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.762312    2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 15 20:04:44.763073 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.762330    2576 state_mem.go:36] "Initialized new in-memory state store"
Sep 15 20:04:44.763073 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.763005    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 20:04:44.763739 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:44.763655    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-155.ec2.internal.18d59735e9d2819b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-155.ec2.internal,UID:ip-10-0-134-155.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-155.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-155.ec2.internal,},FirstTimestamp:2026-09-15 20:04:44.761588123 +0000 UTC m=+0.421799258,LastTimestamp:2026-09-15 20:04:44.761588123 +0000 UTC m=+0.421799258,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-155.ec2.internal,}"
Sep 15 20:04:44.764163 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.764144    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-155.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 20:04:44.764259 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.764198    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-155.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 20:04:44.764259 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.764212    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-155.ec2.internal" event="NodeHasSufficientPID"
Sep 15 20:04:44.764600 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.764585    2576 policy_none.go:49] "None policy: Start"
Sep 15 20:04:44.764658 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.764606    2576 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 15 20:04:44.764658 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.764621    2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 15 20:04:44.766595 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.766581    2576 policy_none.go:47] "Start"
Sep 15 20:04:44.766813 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.766798    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qhbnv"
Sep 15 20:04:44.773112 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:44.773037    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-155.ec2.internal.18d59735e9d2d4be  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-155.ec2.internal,UID:ip-10-0-134-155.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-134-155.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-134-155.ec2.internal,},FirstTimestamp:2026-09-15 20:04:44.761609406 +0000 UTC m=+0.421820534,LastTimestamp:2026-09-15 20:04:44.761609406 +0000 UTC m=+0.421820534,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-155.ec2.internal,}"
Sep 15 20:04:44.774583 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.774568    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qhbnv"
Sep 15 20:04:44.809774 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.809752    2576 manager.go:337] "Starting Device Plugin manager"
Sep 15 20:04:44.809838 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:44.809800    2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 15 20:04:44.809838 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.809812    2576 server.go:87] "Starting device plugin registration server"
Sep 15 20:04:44.810071 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.810057    2576 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 15 20:04:44.810071 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.810071    2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 15 20:04:44.810219 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.810125    2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 15 20:04:44.810308 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.810270    2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 15 20:04:44.810308 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.810282    2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 15 20:04:44.810879 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:44.810856    2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 15 20:04:44.810950 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:44.810901    2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-155.ec2.internal\" not found"
Sep 15 20:04:44.823453 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.823437    2576 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2606 into the system container \"/system.slice\": <nil>"
Sep 15 20:04:44.910988 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.910925    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 20:04:44.911931 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.911913    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-155.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 20:04:44.912009 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.911949    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-155.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 20:04:44.912009 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.911974    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-155.ec2.internal" event="NodeHasSufficientPID"
Sep 15 20:04:44.912009 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.912007    2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-155.ec2.internal"
Sep 15 20:04:44.922286 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.922271    2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-155.ec2.internal"
Sep 15 20:04:44.940016 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:44.922293    2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-155.ec2.internal\": node \"ip-10-0-134-155.ec2.internal\" not found"
Sep 15 20:04:44.947199 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:44.947164    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-155.ec2.internal\" not found"
Sep 15 20:04:44.963593 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.963574    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 15 20:04:44.964820 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.964801    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 15 20:04:44.964886 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.964844    2576 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 15 20:04:44.964886 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.964863    2576 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 15 20:04:44.964963 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:44.964896    2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 15 20:04:44.967861 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:44.967842    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 20:04:45.048093 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:45.048075    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-155.ec2.internal\" not found"
Sep 15 20:04:45.065241 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:45.065215    2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-155.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-155.ec2.internal"]
Sep 15 20:04:45.065296 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:45.065285    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 20:04:45.066212 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:45.066197    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-155.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 20:04:45.066280 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:45.066240    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-155.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 20:04:45.066280 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:45.066250    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-155.ec2.internal" event="NodeHasSufficientPID"
Sep 15 20:04:45.068529 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:45.068517    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 20:04:45.068694 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:45.068679    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-155.ec2.internal"
Sep 15 20:04:45.068733 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:45.068710    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 20:04:45.069387 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:45.069371    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-155.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 20:04:45.069455 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:45.069411    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-155.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 20:04:45.069455 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:45.069430    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-155.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 20:04:45.069455 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:45.069442    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-155.ec2.internal" event="NodeHasSufficientPID"
Sep 15 20:04:45.069590 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:45.069412    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-155.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 20:04:45.069590 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:45.069509    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-155.ec2.internal" event="NodeHasSufficientPID"
Sep 15 20:04:45.071667 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:45.071647    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-155.ec2.internal"
Sep 15 20:04:45.071733 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:45.071683    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 20:04:45.072461 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:45.072441    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-155.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 20:04:45.072519 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:45.072464    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-155.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 20:04:45.072519 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:45.072475    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-155.ec2.internal" event="NodeHasSufficientPID"
Sep 15 20:04:45.095737 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:45.095720    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-155.ec2.internal\" not found" node="ip-10-0-134-155.ec2.internal"
Sep 15 20:04:45.098941 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:45.098923    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-155.ec2.internal\" not found" node="ip-10-0-134-155.ec2.internal"
Sep 15 20:04:45.143810 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:45.143789    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6cec6a6594652885b2e8d4cf7af01cc1-config\") pod \"kube-apiserver-proxy-ip-10-0-134-155.ec2.internal\" (UID: \"6cec6a6594652885b2e8d4cf7af01cc1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-155.ec2.internal"
Sep 15 20:04:45.143907 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:45.143815    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/207cad37faaa53a9bfb12816947f0fd7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-155.ec2.internal\" (UID: \"207cad37faaa53a9bfb12816947f0fd7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-155.ec2.internal"
Sep 15 20:04:45.143907 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:45.143837    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/207cad37faaa53a9bfb12816947f0fd7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-155.ec2.internal\" (UID: \"207cad37faaa53a9bfb12816947f0fd7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-155.ec2.internal"
Sep 15 20:04:45.148895 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:45.148881    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-155.ec2.internal\" not found"
Sep 15 20:04:45.244197 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:45.244118    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/207cad37faaa53a9bfb12816947f0fd7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-155.ec2.internal\" (UID: \"207cad37faaa53a9bfb12816947f0fd7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-155.ec2.internal"
Sep 15 20:04:45.244197 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:45.244146    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/207cad37faaa53a9bfb12816947f0fd7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-155.ec2.internal\" (UID: \"207cad37faaa53a9bfb12816947f0fd7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-155.ec2.internal"
Sep 15 20:04:45.244324 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:45.244185    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6cec6a6594652885b2e8d4cf7af01cc1-config\") pod \"kube-apiserver-proxy-ip-10-0-134-155.ec2.internal\" (UID: \"6cec6a6594652885b2e8d4cf7af01cc1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-155.ec2.internal"
Sep 15 20:04:45.244324 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:45.244231    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/207cad37faaa53a9bfb12816947f0fd7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-155.ec2.internal\" (UID: \"207cad37faaa53a9bfb12816947f0fd7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-155.ec2.internal"
Sep 15 20:04:45.244324 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:45.244253    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6cec6a6594652885b2e8d4cf7af01cc1-config\") pod \"kube-apiserver-proxy-ip-10-0-134-155.ec2.internal\" (UID: \"6cec6a6594652885b2e8d4cf7af01cc1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-155.ec2.internal"
Sep 15 20:04:45.244324 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:45.244234    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/207cad37faaa53a9bfb12816947f0fd7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-155.ec2.internal\" (UID: \"207cad37faaa53a9bfb12816947f0fd7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-155.ec2.internal"
Sep 15 20:04:45.249231 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:45.249217    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-155.ec2.internal\" not found"
Sep 15 20:04:45.349792 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:45.349767    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-155.ec2.internal\" not found"
Sep 15 20:04:46.569704 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:46.569673    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-155.ec2.internal"
Sep 15 20:04:46.574450 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:46.574429    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-155.ec2.internal"
Sep 15 20:04:46.622692 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:46.622552    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-155.ec2.internal\" not found"
Sep 15 20:04:46.723037 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:46.722992    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-155.ec2.internal\" not found"
Sep 15 20:04:46.823588 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:46.823550    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-155.ec2.internal\" not found"
Sep 15 20:04:46.846725 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:46.846705    2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 15 20:04:46.847118 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:46.846840    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 15 20:04:46.923736 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:46.923706    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-155.ec2.internal\" not found"
Sep 15 20:04:46.925698 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:46.925677    2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 20:04:46.951861 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:46.951823    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-14 19:59:45 +0000 UTC" deadline="2028-03-31 03:31:00.109148537 +0000 UTC"
Sep 15 20:04:46.951861 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:46.951852    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13495h26m13.157300037s"
Sep 15 20:04:47.011532 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.011483    2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 20:04:47.013343 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.013329    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-155.ec2.internal"
Sep 15 20:04:47.036342 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.036328    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 15 20:04:47.037296 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.037285    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-155.ec2.internal"
Sep 15 20:04:47.045258 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.045246    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 15 20:04:47.059890 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.059873    2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 20:04:47.440295 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:47.440266    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod207cad37faaa53a9bfb12816947f0fd7.slice/crio-36cd4d5896729bb6d68e4e7b41b362f02728f9bebfaf416594fc18175e7bd614 WatchSource:0}: Error finding container 36cd4d5896729bb6d68e4e7b41b362f02728f9bebfaf416594fc18175e7bd614: Status 404 returned error can't find the container with id 36cd4d5896729bb6d68e4e7b41b362f02728f9bebfaf416594fc18175e7bd614
Sep 15 20:04:47.440695 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:47.440679    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cec6a6594652885b2e8d4cf7af01cc1.slice/crio-1ca2b9ae04fe1ec86c2faa917d95d5308f1f6b89726668ff53f0c9e46173e9b5 WatchSource:0}: Error finding container 1ca2b9ae04fe1ec86c2faa917d95d5308f1f6b89726668ff53f0c9e46173e9b5: Status 404 returned error can't find the container with id 1ca2b9ae04fe1ec86c2faa917d95d5308f1f6b89726668ff53f0c9e46173e9b5
Sep 15 20:04:47.446755 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.446738    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 20:04:47.887784 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.887760    2576 apiserver.go:52] "Watching apiserver"
Sep 15 20:04:47.893636 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.893620    2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 15 20:04:47.893995 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.893976    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-134-155.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qm42p","openshift-cluster-node-tuning-operator/tuned-ldvcx","openshift-dns/node-resolver-s8ddr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-155.ec2.internal","openshift-multus/network-metrics-daemon-59mfs","openshift-network-diagnostics/network-check-target-gzw9m","kube-system/global-pull-secret-syncer-7hmwc","kube-system/konnectivity-agent-4vdw2","openshift-image-registry/node-ca-qk9ld","openshift-multus/multus-additional-cni-plugins-9r4qs","openshift-multus/multus-f7npj","openshift-network-operator/iptables-alerter-rs6ws","openshift-ovn-kubernetes/ovnkube-node-6st24"]
Sep 15 20:04:47.897638 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.897621    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzw9m"
Sep 15 20:04:47.897729 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:47.897707    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gzw9m" podUID="43a5c998-df74-4a93-b02f-3d53c8be805a"
Sep 15 20:04:47.899627 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.899609    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9r4qs"
Sep 15 20:04:47.901754 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.901730    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:47.902019 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.901995    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 15 20:04:47.902149 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.902096    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 15 20:04:47.902149 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.902107    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 15 20:04:47.902261 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.902179    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 15 20:04:47.902261 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.902109    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 15 20:04:47.902261 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.902251    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-vws64\""
Sep 15 20:04:47.903790 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.903772    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-s8ddr"
Sep 15 20:04:47.903989 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.903972    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 15 20:04:47.904199 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.904185    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-95s7m\""
Sep 15 20:04:47.904523 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.904511    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 15 20:04:47.905730 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.905716    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-r4hcp\""
Sep 15 20:04:47.905972 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.905959    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 15 20:04:47.906211 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.906095    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 15 20:04:47.906294 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.906278    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:47.910898 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.908419    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 15 20:04:47.910898 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.908869    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 15 20:04:47.910898 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.908910    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 15 20:04:47.910898 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.909293    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 15 20:04:47.910898 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.909423    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 15 20:04:47.910898 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.909412    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-lfxsw\""
Sep 15 20:04:47.910898 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.909466    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 15 20:04:47.910898 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.909679    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:04:47.910898 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:47.909782    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59mfs" podUID="3b8d9342-6574-4d0d-a326-0299eb3b4c5d"
Sep 15 20:04:47.911628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.911609    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4vdw2"
Sep 15 20:04:47.913047 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.913031    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 15 20:04:47.913574 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.913558    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9ssgz\""
Sep 15 20:04:47.913670 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.913615    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 15 20:04:47.913770 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.913744    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 15 20:04:47.913876 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.913858    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rs6ws"
Sep 15 20:04:47.915788 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.915771    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 15 20:04:47.915893 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.915871    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7sdhq\""
Sep 15 20:04:47.915974 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.915882    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 15 20:04:47.916033 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.915986    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 15 20:04:47.916164 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.916152    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qm42p"
Sep 15 20:04:47.918202 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.918177    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 15 20:04:47.918351 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.918332    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-chq7b\""
Sep 15 20:04:47.918425 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.918412    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 15 20:04:47.918588 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.918573    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 15 20:04:47.918657 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.918634    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qk9ld"
Sep 15 20:04:47.920655 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.920630    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 15 20:04:47.920655 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.920620    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 15 20:04:47.920873 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.920856    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7hmwc"
Sep 15 20:04:47.920951 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.920890    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4tbcc\""
Sep 15 20:04:47.920951 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:47.920911    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7hmwc" podUID="6c3054ad-31af-4d06-9809-1eab5e2b140c"
Sep 15 20:04:47.920951 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.920890    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 15 20:04:47.923083 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.923066    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-f7npj"
Sep 15 20:04:47.925411 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.925397    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 15 20:04:47.925559 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.925549    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-t4whs\""
Sep 15 20:04:47.925891 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.925800    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-host-cni-netd\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:47.925891 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.925806    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 15 20:04:47.925891 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.925832    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/794a8849-e693-4fd1-905d-addff4740cda-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9r4qs\" (UID: \"794a8849-e693-4fd1-905d-addff4740cda\") " pod="openshift-multus/multus-additional-cni-plugins-9r4qs"
Sep 15 20:04:47.926086 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.925895    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/25927e9f-cdb1-49d8-8d63-ff66a3678f87-etc-sysconfig\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:47.926086 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.925938    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/25927e9f-cdb1-49d8-8d63-ff66a3678f87-host\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:47.926086 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.925967    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-run-systemd\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:47.926086 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926006    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-node-log\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:47.926086 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926036    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e6bec389-ca68-4845-8694-9571cf84fe3f-konnectivity-ca\") pod \"konnectivity-agent-4vdw2\" (UID: \"e6bec389-ca68-4845-8694-9571cf84fe3f\") " pod="kube-system/konnectivity-agent-4vdw2"
Sep 15 20:04:47.926086 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926074    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-metrics-certs\") pod \"network-metrics-daemon-59mfs\" (UID: \"3b8d9342-6574-4d0d-a326-0299eb3b4c5d\") " pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:04:47.926311 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926098    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/25927e9f-cdb1-49d8-8d63-ff66a3678f87-etc-systemd\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:47.926311 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926120    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/25927e9f-cdb1-49d8-8d63-ff66a3678f87-etc-tuned\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:47.926311 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926166    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-var-lib-openvswitch\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:47.926311 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926189    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-run-ovn\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:47.926311 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926212    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-host-run-ovn-kubernetes\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:47.926311 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926243    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-env-overrides\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:47.926311 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926276    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmzwk\" (UniqueName: \"kubernetes.io/projected/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-kube-api-access-vmzwk\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:47.926311 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926310    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/55164773-0e7d-48b1-b300-2f08b68ed675-iptables-alerter-script\") pod \"iptables-alerter-rs6ws\" (UID: \"55164773-0e7d-48b1-b300-2f08b68ed675\") " pod="openshift-network-operator/iptables-alerter-rs6ws"
Sep 15 20:04:47.926574 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926330    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/794a8849-e693-4fd1-905d-addff4740cda-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9r4qs\" (UID: \"794a8849-e693-4fd1-905d-addff4740cda\") " pod="openshift-multus/multus-additional-cni-plugins-9r4qs"
Sep 15 20:04:47.926574 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926352    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25927e9f-cdb1-49d8-8d63-ff66a3678f87-etc-kubernetes\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:47.926574 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926374    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/25927e9f-cdb1-49d8-8d63-ff66a3678f87-tmp\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:47.926574 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926402    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6cd01117-433d-4273-a167-01344ad4f018-hosts-file\") pod \"node-resolver-s8ddr\" (UID: \"6cd01117-433d-4273-a167-01344ad4f018\") " pod="openshift-dns/node-resolver-s8ddr"
Sep 15 20:04:47.926574 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926417    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-host-cni-bin\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:47.926574 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926450    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:47.926574 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926473    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e6bec389-ca68-4845-8694-9571cf84fe3f-agent-certs\") pod \"konnectivity-agent-4vdw2\" (UID: \"e6bec389-ca68-4845-8694-9571cf84fe3f\") " pod="kube-system/konnectivity-agent-4vdw2"
Sep 15 20:04:47.926574 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926496    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqdjp\" (UniqueName: \"kubernetes.io/projected/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-kube-api-access-mqdjp\") pod \"network-metrics-daemon-59mfs\" (UID: \"3b8d9342-6574-4d0d-a326-0299eb3b4c5d\") " pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:04:47.926574 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926517    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/794a8849-e693-4fd1-905d-addff4740cda-system-cni-dir\") pod \"multus-additional-cni-plugins-9r4qs\" (UID: \"794a8849-e693-4fd1-905d-addff4740cda\") " pod="openshift-multus/multus-additional-cni-plugins-9r4qs"
Sep 15 20:04:47.926574 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926558    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/25927e9f-cdb1-49d8-8d63-ff66a3678f87-etc-sysctl-d\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:47.926895 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926578    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/25927e9f-cdb1-49d8-8d63-ff66a3678f87-lib-modules\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:47.926895 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926592    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsh7l\" (UniqueName: \"kubernetes.io/projected/6cd01117-433d-4273-a167-01344ad4f018-kube-api-access-dsh7l\") pod \"node-resolver-s8ddr\" (UID: \"6cd01117-433d-4273-a167-01344ad4f018\") " pod="openshift-dns/node-resolver-s8ddr"
Sep 15 20:04:47.926895 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926604    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/55164773-0e7d-48b1-b300-2f08b68ed675-host-slash\") pod \"iptables-alerter-rs6ws\" (UID: \"55164773-0e7d-48b1-b300-2f08b68ed675\") " pod="openshift-network-operator/iptables-alerter-rs6ws"
Sep 15 20:04:47.926895 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926639    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/794a8849-e693-4fd1-905d-addff4740cda-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9r4qs\" (UID: \"794a8849-e693-4fd1-905d-addff4740cda\") " pod="openshift-multus/multus-additional-cni-plugins-9r4qs"
Sep 15 20:04:47.926895 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926658    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksqbk\" (UniqueName: \"kubernetes.io/projected/794a8849-e693-4fd1-905d-addff4740cda-kube-api-access-ksqbk\") pod \"multus-additional-cni-plugins-9r4qs\" (UID: \"794a8849-e693-4fd1-905d-addff4740cda\") " pod="openshift-multus/multus-additional-cni-plugins-9r4qs"
Sep 15 20:04:47.926895 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926672    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-host-slash\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:47.926895 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926688    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-host-run-netns\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:47.926895 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926708    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-ovnkube-config\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:47.926895 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926732    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-ovn-node-metrics-cert\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:47.926895 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926750    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx8w7\" (UniqueName: \"kubernetes.io/projected/55164773-0e7d-48b1-b300-2f08b68ed675-kube-api-access-nx8w7\") pod \"iptables-alerter-rs6ws\" (UID: \"55164773-0e7d-48b1-b300-2f08b68ed675\") " pod="openshift-network-operator/iptables-alerter-rs6ws"
Sep 15 20:04:47.926895 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926774    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/25927e9f-cdb1-49d8-8d63-ff66a3678f87-etc-modprobe-d\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:47.926895 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926814    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/25927e9f-cdb1-49d8-8d63-ff66a3678f87-run\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:47.926895 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926845    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-systemd-units\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:47.926895 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926873    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-etc-openvswitch\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:47.927394 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926900    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/794a8849-e693-4fd1-905d-addff4740cda-cnibin\") pod \"multus-additional-cni-plugins-9r4qs\" (UID: \"794a8849-e693-4fd1-905d-addff4740cda\") " pod="openshift-multus/multus-additional-cni-plugins-9r4qs"
Sep 15 20:04:47.927394 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926941    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/25927e9f-cdb1-49d8-8d63-ff66a3678f87-etc-sysctl-conf\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:47.927394 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.926984    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/25927e9f-cdb1-49d8-8d63-ff66a3678f87-var-lib-kubelet\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:47.927394 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.927020    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-run-openvswitch\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:47.927394 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.927038    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-ovnkube-script-lib\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:47.927394 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.927054    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sblk8\" (UniqueName: \"kubernetes.io/projected/43a5c998-df74-4a93-b02f-3d53c8be805a-kube-api-access-sblk8\") pod \"network-check-target-gzw9m\" (UID: \"43a5c998-df74-4a93-b02f-3d53c8be805a\") " pod="openshift-network-diagnostics/network-check-target-gzw9m"
Sep 15 20:04:47.927394 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.927077    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/794a8849-e693-4fd1-905d-addff4740cda-cni-binary-copy\") pod \"multus-additional-cni-plugins-9r4qs\" (UID: \"794a8849-e693-4fd1-905d-addff4740cda\") " pod="openshift-multus/multus-additional-cni-plugins-9r4qs"
Sep 15 20:04:47.927394 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.927102    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25927e9f-cdb1-49d8-8d63-ff66a3678f87-sys\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:47.927394 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.927130    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmt8g\" (UniqueName: \"kubernetes.io/projected/25927e9f-cdb1-49d8-8d63-ff66a3678f87-kube-api-access-wmt8g\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:47.927394 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.927170    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cd01117-433d-4273-a167-01344ad4f018-tmp-dir\") pod \"node-resolver-s8ddr\" (UID: \"6cd01117-433d-4273-a167-01344ad4f018\") " pod="openshift-dns/node-resolver-s8ddr"
Sep 15 20:04:47.927394 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.927212    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-host-kubelet\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:47.927394 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.927234    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-log-socket\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:47.943549 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.943534    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6b8wl"
Sep 15 20:04:47.950059 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:47.950043    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6b8wl"
Sep 15 20:04:48.015079 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.015060    2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 15 20:04:48.027771 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.027747    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6940cbba-826b-4e63-97cd-882dc35fdc9c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qm42p\" (UID: \"6940cbba-826b-4e63-97cd-882dc35fdc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qm42p"
Sep 15 20:04:48.027864 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.027785    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6940cbba-826b-4e63-97cd-882dc35fdc9c-sys-fs\") pod \"aws-ebs-csi-driver-node-qm42p\" (UID: \"6940cbba-826b-4e63-97cd-882dc35fdc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qm42p"
Sep 15 20:04:48.027864 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.027811    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/794a8849-e693-4fd1-905d-addff4740cda-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9r4qs\" (UID: \"794a8849-e693-4fd1-905d-addff4740cda\") " pod="openshift-multus/multus-additional-cni-plugins-9r4qs"
Sep 15 20:04:48.027952 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.027908    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/25927e9f-cdb1-49d8-8d63-ff66a3678f87-etc-sysconfig\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:48.027991 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.027961    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/794a8849-e693-4fd1-905d-addff4740cda-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9r4qs\" (UID: \"794a8849-e693-4fd1-905d-addff4740cda\") " pod="openshift-multus/multus-additional-cni-plugins-9r4qs"
Sep 15 20:04:48.027991 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.027976    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/25927e9f-cdb1-49d8-8d63-ff66a3678f87-etc-sysconfig\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:48.028060 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.027987    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/25927e9f-cdb1-49d8-8d63-ff66a3678f87-host\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:48.028060 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.028009    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e6bec389-ca68-4845-8694-9571cf84fe3f-konnectivity-ca\") pod \"konnectivity-agent-4vdw2\" (UID: \"e6bec389-ca68-4845-8694-9571cf84fe3f\") " pod="kube-system/konnectivity-agent-4vdw2"
Sep 15 20:04:48.028060 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.028023    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-metrics-certs\") pod \"network-metrics-daemon-59mfs\" (UID: \"3b8d9342-6574-4d0d-a326-0299eb3b4c5d\") " pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:04:48.028060 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.028036    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6c3054ad-31af-4d06-9809-1eab5e2b140c-dbus\") pod \"global-pull-secret-syncer-7hmwc\" (UID: \"6c3054ad-31af-4d06-9809-1eab5e2b140c\") " pod="kube-system/global-pull-secret-syncer-7hmwc"
Sep 15 20:04:48.028218 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.028049    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-system-cni-dir\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.028218 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.028117    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np6m9\" (UniqueName: \"kubernetes.io/projected/5368fcd5-c6e8-4f29-b93c-338fdd857bad-kube-api-access-np6m9\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.028218 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:48.028149    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:04:48.028218 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:48.028216    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-metrics-certs podName:3b8d9342-6574-4d0d-a326-0299eb3b4c5d nodeName:}" failed. No retries permitted until 2026-09-15 20:04:48.528201004 +0000 UTC m=+3.016665457 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-metrics-certs") pod "network-metrics-daemon-59mfs" (UID: "3b8d9342-6574-4d0d-a326-0299eb3b4c5d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:04:48.028372 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.028260    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/25927e9f-cdb1-49d8-8d63-ff66a3678f87-host\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:48.028372 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.028277    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/25927e9f-cdb1-49d8-8d63-ff66a3678f87-etc-systemd\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:48.028372 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.028308    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/25927e9f-cdb1-49d8-8d63-ff66a3678f87-etc-tuned\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:48.028372 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.028322    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-var-lib-openvswitch\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.028372 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.028342    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-host-run-ovn-kubernetes\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.028372 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.028348    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/25927e9f-cdb1-49d8-8d63-ff66a3678f87-etc-systemd\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:48.028610 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.028392    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-host-run-ovn-kubernetes\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.028610 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.028448    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-var-lib-openvswitch\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.028610 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.028454    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmzwk\" (UniqueName: \"kubernetes.io/projected/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-kube-api-access-vmzwk\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.029346 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.029104    2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 15 20:04:48.029346 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.029273    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/55164773-0e7d-48b1-b300-2f08b68ed675-iptables-alerter-script\") pod \"iptables-alerter-rs6ws\" (UID: \"55164773-0e7d-48b1-b300-2f08b68ed675\") " pod="openshift-network-operator/iptables-alerter-rs6ws"
Sep 15 20:04:48.029346 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.028725    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e6bec389-ca68-4845-8694-9571cf84fe3f-konnectivity-ca\") pod \"konnectivity-agent-4vdw2\" (UID: \"e6bec389-ca68-4845-8694-9571cf84fe3f\") " pod="kube-system/konnectivity-agent-4vdw2"
Sep 15 20:04:48.029535 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.029430    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6940cbba-826b-4e63-97cd-882dc35fdc9c-socket-dir\") pod \"aws-ebs-csi-driver-node-qm42p\" (UID: \"6940cbba-826b-4e63-97cd-882dc35fdc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qm42p"
Sep 15 20:04:48.029535 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.029470    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6940cbba-826b-4e63-97cd-882dc35fdc9c-etc-selinux\") pod \"aws-ebs-csi-driver-node-qm42p\" (UID: \"6940cbba-826b-4e63-97cd-882dc35fdc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qm42p"
Sep 15 20:04:48.029535 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.029507    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/794a8849-e693-4fd1-905d-addff4740cda-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9r4qs\" (UID: \"794a8849-e693-4fd1-905d-addff4740cda\") " pod="openshift-multus/multus-additional-cni-plugins-9r4qs"
Sep 15 20:04:48.029535 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.029533    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25927e9f-cdb1-49d8-8d63-ff66a3678f87-etc-kubernetes\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:48.029733 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.029551    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/55164773-0e7d-48b1-b300-2f08b68ed675-iptables-alerter-script\") pod \"iptables-alerter-rs6ws\" (UID: \"55164773-0e7d-48b1-b300-2f08b68ed675\") " pod="openshift-network-operator/iptables-alerter-rs6ws"
Sep 15 20:04:48.029733 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.029563    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/25927e9f-cdb1-49d8-8d63-ff66a3678f87-tmp\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:48.029733 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.029594    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e6bec389-ca68-4845-8694-9571cf84fe3f-agent-certs\") pod \"konnectivity-agent-4vdw2\" (UID: \"e6bec389-ca68-4845-8694-9571cf84fe3f\") " pod="kube-system/konnectivity-agent-4vdw2"
Sep 15 20:04:48.029733 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.029624    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqdjp\" (UniqueName: \"kubernetes.io/projected/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-kube-api-access-mqdjp\") pod \"network-metrics-daemon-59mfs\" (UID: \"3b8d9342-6574-4d0d-a326-0299eb3b4c5d\") " pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:04:48.029733 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.029655    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-host-var-lib-cni-bin\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.029733 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.029687    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/25927e9f-cdb1-49d8-8d63-ff66a3678f87-etc-sysctl-d\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:48.029733 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.029720    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/25927e9f-cdb1-49d8-8d63-ff66a3678f87-lib-modules\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:48.030073 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.029759    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsh7l\" (UniqueName: \"kubernetes.io/projected/6cd01117-433d-4273-a167-01344ad4f018-kube-api-access-dsh7l\") pod \"node-resolver-s8ddr\" (UID: \"6cd01117-433d-4273-a167-01344ad4f018\") " pod="openshift-dns/node-resolver-s8ddr"
Sep 15 20:04:48.030073 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.029789    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c98sm\" (UniqueName: \"kubernetes.io/projected/48d7dc57-6de6-4a72-8da3-156c90d7a8ff-kube-api-access-c98sm\") pod \"node-ca-qk9ld\" (UID: \"48d7dc57-6de6-4a72-8da3-156c90d7a8ff\") " pod="openshift-image-registry/node-ca-qk9ld"
Sep 15 20:04:48.030073 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.029813    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-multus-socket-dir-parent\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.030073 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.029935    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-etc-kubernetes\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.030073 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.029975    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/25927e9f-cdb1-49d8-8d63-ff66a3678f87-etc-modprobe-d\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:48.030073 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.029981    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/25927e9f-cdb1-49d8-8d63-ff66a3678f87-lib-modules\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:48.030073 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.030013    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/25927e9f-cdb1-49d8-8d63-ff66a3678f87-run\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:48.030073 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.030046    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-systemd-units\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.030431 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.030079    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25927e9f-cdb1-49d8-8d63-ff66a3678f87-etc-kubernetes\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:48.030431 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.030084    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-multus-cni-dir\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.030431 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.030137    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-host-var-lib-cni-multus\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.030431 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.030087    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/25927e9f-cdb1-49d8-8d63-ff66a3678f87-etc-modprobe-d\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:48.030431 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.030149    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-systemd-units\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.030431 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.030265    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/25927e9f-cdb1-49d8-8d63-ff66a3678f87-etc-sysctl-d\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:48.030431 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.030302    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-host-var-lib-kubelet\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.030431 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.030332    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6940cbba-826b-4e63-97cd-882dc35fdc9c-registration-dir\") pod \"aws-ebs-csi-driver-node-qm42p\" (UID: \"6940cbba-826b-4e63-97cd-882dc35fdc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qm42p"
Sep 15 20:04:48.030431 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.030426    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/25927e9f-cdb1-49d8-8d63-ff66a3678f87-etc-sysctl-conf\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:48.030876 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.030455    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/25927e9f-cdb1-49d8-8d63-ff66a3678f87-var-lib-kubelet\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:48.030876 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.030485    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-ovnkube-script-lib\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.030876 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.030519    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6c3054ad-31af-4d06-9809-1eab5e2b140c-kubelet-config\") pod \"global-pull-secret-syncer-7hmwc\" (UID: \"6c3054ad-31af-4d06-9809-1eab5e2b140c\") " pod="kube-system/global-pull-secret-syncer-7hmwc"
Sep 15 20:04:48.030876 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.030584    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-host-run-multus-certs\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.030876 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.030609    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cd01117-433d-4273-a167-01344ad4f018-tmp-dir\") pod \"node-resolver-s8ddr\" (UID: \"6cd01117-433d-4273-a167-01344ad4f018\") " pod="openshift-dns/node-resolver-s8ddr"
Sep 15 20:04:48.030876 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.030638    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-host-kubelet\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.030876 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.030667    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-log-socket\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.030876 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.030696    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-hostroot\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.030876 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.030725    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6940cbba-826b-4e63-97cd-882dc35fdc9c-device-dir\") pod \"aws-ebs-csi-driver-node-qm42p\" (UID: \"6940cbba-826b-4e63-97cd-882dc35fdc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qm42p"
Sep 15 20:04:48.030876 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.030750    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-run-systemd\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.030876 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.030789    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-node-log\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.031336 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.030898    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/48d7dc57-6de6-4a72-8da3-156c90d7a8ff-host\") pod \"node-ca-qk9ld\" (UID: \"48d7dc57-6de6-4a72-8da3-156c90d7a8ff\") " pod="openshift-image-registry/node-ca-qk9ld"
Sep 15 20:04:48.031336 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.030948    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-host-run-netns\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.031336 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.030970    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/25927e9f-cdb1-49d8-8d63-ff66a3678f87-run\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:48.031336 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.030982    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-run-ovn\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.031336 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.031007    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-env-overrides\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.031336 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.031035    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx8w7\" (UniqueName: \"kubernetes.io/projected/55164773-0e7d-48b1-b300-2f08b68ed675-kube-api-access-nx8w7\") pod \"iptables-alerter-rs6ws\" (UID: \"55164773-0e7d-48b1-b300-2f08b68ed675\") " pod="openshift-network-operator/iptables-alerter-rs6ws"
Sep 15 20:04:48.031336 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.031066    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6cd01117-433d-4273-a167-01344ad4f018-hosts-file\") pod \"node-resolver-s8ddr\" (UID: \"6cd01117-433d-4273-a167-01344ad4f018\") " pod="openshift-dns/node-resolver-s8ddr"
Sep 15 20:04:48.031336 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.031094    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-host-cni-bin\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.031336 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.031082    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/25927e9f-cdb1-49d8-8d63-ff66a3678f87-etc-sysctl-conf\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:48.031336 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.031140    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/25927e9f-cdb1-49d8-8d63-ff66a3678f87-var-lib-kubelet\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:48.031336 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.031151    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-node-log\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.031336 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.031174    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.031336 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.031214    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-run-ovn\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.031336 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.031213    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/55164773-0e7d-48b1-b300-2f08b68ed675-host-slash\") pod \"iptables-alerter-rs6ws\" (UID: \"55164773-0e7d-48b1-b300-2f08b68ed675\") " pod="openshift-network-operator/iptables-alerter-rs6ws"
Sep 15 20:04:48.031336 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.031247    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c3054ad-31af-4d06-9809-1eab5e2b140c-original-pull-secret\") pod \"global-pull-secret-syncer-7hmwc\" (UID: \"6c3054ad-31af-4d06-9809-1eab5e2b140c\") " pod="kube-system/global-pull-secret-syncer-7hmwc"
Sep 15 20:04:48.031336 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.031284    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/794a8849-e693-4fd1-905d-addff4740cda-system-cni-dir\") pod \"multus-additional-cni-plugins-9r4qs\" (UID: \"794a8849-e693-4fd1-905d-addff4740cda\") " pod="openshift-multus/multus-additional-cni-plugins-9r4qs"
Sep 15 20:04:48.032064 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.031357    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/48d7dc57-6de6-4a72-8da3-156c90d7a8ff-serviceca\") pod \"node-ca-qk9ld\" (UID: \"48d7dc57-6de6-4a72-8da3-156c90d7a8ff\") " pod="openshift-image-registry/node-ca-qk9ld"
Sep 15 20:04:48.032064 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.031381    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-cnibin\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.032064 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.031417    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5368fcd5-c6e8-4f29-b93c-338fdd857bad-cni-binary-copy\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.032064 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.031461    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-host-run-k8s-cni-cncf-io\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.032064 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.031496    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/794a8849-e693-4fd1-905d-addff4740cda-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9r4qs\" (UID: \"794a8849-e693-4fd1-905d-addff4740cda\") " pod="openshift-multus/multus-additional-cni-plugins-9r4qs"
Sep 15 20:04:48.032064 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.031529    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksqbk\" (UniqueName: \"kubernetes.io/projected/794a8849-e693-4fd1-905d-addff4740cda-kube-api-access-ksqbk\") pod \"multus-additional-cni-plugins-9r4qs\" (UID: \"794a8849-e693-4fd1-905d-addff4740cda\") " pod="openshift-multus/multus-additional-cni-plugins-9r4qs"
Sep 15 20:04:48.032064 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.031556    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-host-slash\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.032064 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.031586    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-host-run-netns\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.032064 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.031614    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-ovnkube-config\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.032064 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.031645    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-ovn-node-metrics-cert\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.032064 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.031675    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-etc-openvswitch\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.032064 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.031705    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/794a8849-e693-4fd1-905d-addff4740cda-cnibin\") pod \"multus-additional-cni-plugins-9r4qs\" (UID: \"794a8849-e693-4fd1-905d-addff4740cda\") " pod="openshift-multus/multus-additional-cni-plugins-9r4qs"
Sep 15 20:04:48.032064 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.031674    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-env-overrides\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.032064 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.031836    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-run-openvswitch\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.032064 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.031843    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-etc-openvswitch\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.032064 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.031863    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sblk8\" (UniqueName: \"kubernetes.io/projected/43a5c998-df74-4a93-b02f-3d53c8be805a-kube-api-access-sblk8\") pod \"network-check-target-gzw9m\" (UID: \"43a5c998-df74-4a93-b02f-3d53c8be805a\") " pod="openshift-network-diagnostics/network-check-target-gzw9m"
Sep 15 20:04:48.032064 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.031895    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5368fcd5-c6e8-4f29-b93c-338fdd857bad-multus-daemon-config\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.032868 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.031953    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gm5tm\" (UniqueName: \"kubernetes.io/projected/6940cbba-826b-4e63-97cd-882dc35fdc9c-kube-api-access-gm5tm\") pod \"aws-ebs-csi-driver-node-qm42p\" (UID: \"6940cbba-826b-4e63-97cd-882dc35fdc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qm42p"
Sep 15 20:04:48.032868 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.031985    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/794a8849-e693-4fd1-905d-addff4740cda-cni-binary-copy\") pod \"multus-additional-cni-plugins-9r4qs\" (UID: \"794a8849-e693-4fd1-905d-addff4740cda\") " pod="openshift-multus/multus-additional-cni-plugins-9r4qs"
Sep 15 20:04:48.032868 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.032016    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25927e9f-cdb1-49d8-8d63-ff66a3678f87-sys\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:48.032868 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.032041    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmt8g\" (UniqueName: \"kubernetes.io/projected/25927e9f-cdb1-49d8-8d63-ff66a3678f87-kube-api-access-wmt8g\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:48.032868 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.032071    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-host-cni-netd\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.032868 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.032100    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-multus-conf-dir\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.032868 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.032132    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-host-slash\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.032868 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.032138    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-host-run-netns\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.032868 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.032534    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/794a8849-e693-4fd1-905d-addff4740cda-system-cni-dir\") pod \"multus-additional-cni-plugins-9r4qs\" (UID: \"794a8849-e693-4fd1-905d-addff4740cda\") " pod="openshift-multus/multus-additional-cni-plugins-9r4qs"
Sep 15 20:04:48.032868 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.032531    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cd01117-433d-4273-a167-01344ad4f018-tmp-dir\") pod \"node-resolver-s8ddr\" (UID: \"6cd01117-433d-4273-a167-01344ad4f018\") " pod="openshift-dns/node-resolver-s8ddr"
Sep 15 20:04:48.032868 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.032590    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-run-openvswitch\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.032868 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.032858    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/794a8849-e693-4fd1-905d-addff4740cda-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9r4qs\" (UID: \"794a8849-e693-4fd1-905d-addff4740cda\") " pod="openshift-multus/multus-additional-cni-plugins-9r4qs"
Sep 15 20:04:48.033449 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.032942    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-ovnkube-script-lib\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.033449 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.033404    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6cd01117-433d-4273-a167-01344ad4f018-hosts-file\") pod \"node-resolver-s8ddr\" (UID: \"6cd01117-433d-4273-a167-01344ad4f018\") " pod="openshift-dns/node-resolver-s8ddr"
Sep 15 20:04:48.033449 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.033429    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-ovnkube-config\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.033581 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.033484    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/794a8849-e693-4fd1-905d-addff4740cda-cnibin\") pod \"multus-additional-cni-plugins-9r4qs\" (UID: \"794a8849-e693-4fd1-905d-addff4740cda\") " pod="openshift-multus/multus-additional-cni-plugins-9r4qs"
Sep 15 20:04:48.033581 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.033530    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-log-socket\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.033581 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.033548    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/794a8849-e693-4fd1-905d-addff4740cda-cni-binary-copy\") pod \"multus-additional-cni-plugins-9r4qs\" (UID: \"794a8849-e693-4fd1-905d-addff4740cda\") " pod="openshift-multus/multus-additional-cni-plugins-9r4qs"
Sep 15 20:04:48.033581 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.033571    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25927e9f-cdb1-49d8-8d63-ff66a3678f87-sys\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:48.033764 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.033624    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-host-kubelet\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.033974 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.033952    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-run-systemd\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.034063 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.034032    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/55164773-0e7d-48b1-b300-2f08b68ed675-host-slash\") pod \"iptables-alerter-rs6ws\" (UID: \"55164773-0e7d-48b1-b300-2f08b68ed675\") " pod="openshift-network-operator/iptables-alerter-rs6ws"
Sep 15 20:04:48.034456 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.034067    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/794a8849-e693-4fd1-905d-addff4740cda-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9r4qs\" (UID: \"794a8849-e693-4fd1-905d-addff4740cda\") " pod="openshift-multus/multus-additional-cni-plugins-9r4qs"
Sep 15 20:04:48.034456 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.034139    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-host-cni-bin\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.034456 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.034186    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.034456 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.034227    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-host-cni-netd\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.034456 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.034376    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/25927e9f-cdb1-49d8-8d63-ff66a3678f87-tmp\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:48.034780 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.034761    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/25927e9f-cdb1-49d8-8d63-ff66a3678f87-etc-tuned\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:48.035599 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.035577    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e6bec389-ca68-4845-8694-9571cf84fe3f-agent-certs\") pod \"konnectivity-agent-4vdw2\" (UID: \"e6bec389-ca68-4845-8694-9571cf84fe3f\") " pod="kube-system/konnectivity-agent-4vdw2"
Sep 15 20:04:48.036300 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.036281    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-ovn-node-metrics-cert\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.039691 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.039670    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsh7l\" (UniqueName: \"kubernetes.io/projected/6cd01117-433d-4273-a167-01344ad4f018-kube-api-access-dsh7l\") pod \"node-resolver-s8ddr\" (UID: \"6cd01117-433d-4273-a167-01344ad4f018\") " pod="openshift-dns/node-resolver-s8ddr"
Sep 15 20:04:48.039802 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.039781    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmzwk\" (UniqueName: \"kubernetes.io/projected/6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e-kube-api-access-vmzwk\") pod \"ovnkube-node-6st24\" (UID: \"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.040575 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:48.040560    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 20:04:48.040652 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:48.040580    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 20:04:48.040652 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:48.040592    2576 projected.go:196] Error preparing data for projected volume kube-api-access-sblk8 for pod openshift-network-diagnostics/network-check-target-gzw9m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:04:48.040652 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:48.040646    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43a5c998-df74-4a93-b02f-3d53c8be805a-kube-api-access-sblk8 podName:43a5c998-df74-4a93-b02f-3d53c8be805a nodeName:}" failed. No retries permitted until 2026-09-15 20:04:48.540629555 +0000 UTC m=+3.029094014 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-sblk8" (UniqueName: "kubernetes.io/projected/43a5c998-df74-4a93-b02f-3d53c8be805a-kube-api-access-sblk8") pod "network-check-target-gzw9m" (UID: "43a5c998-df74-4a93-b02f-3d53c8be805a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:04:48.040972 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.040949    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx8w7\" (UniqueName: \"kubernetes.io/projected/55164773-0e7d-48b1-b300-2f08b68ed675-kube-api-access-nx8w7\") pod \"iptables-alerter-rs6ws\" (UID: \"55164773-0e7d-48b1-b300-2f08b68ed675\") " pod="openshift-network-operator/iptables-alerter-rs6ws"
Sep 15 20:04:48.042225 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.042209    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksqbk\" (UniqueName: \"kubernetes.io/projected/794a8849-e693-4fd1-905d-addff4740cda-kube-api-access-ksqbk\") pod \"multus-additional-cni-plugins-9r4qs\" (UID: \"794a8849-e693-4fd1-905d-addff4740cda\") " pod="openshift-multus/multus-additional-cni-plugins-9r4qs"
Sep 15 20:04:48.045112 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.045096    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmt8g\" (UniqueName: \"kubernetes.io/projected/25927e9f-cdb1-49d8-8d63-ff66a3678f87-kube-api-access-wmt8g\") pod \"tuned-ldvcx\" (UID: \"25927e9f-cdb1-49d8-8d63-ff66a3678f87\") " pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:48.045295 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.045280    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 20:04:48.045524 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.045509    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqdjp\" (UniqueName: \"kubernetes.io/projected/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-kube-api-access-mqdjp\") pod \"network-metrics-daemon-59mfs\" (UID: \"3b8d9342-6574-4d0d-a326-0299eb3b4c5d\") " pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:04:48.132932 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.132900    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-multus-cni-dir\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.132932 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.132940    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-host-var-lib-cni-multus\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.133129 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.132955    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-host-var-lib-kubelet\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.133129 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.132971    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6940cbba-826b-4e63-97cd-882dc35fdc9c-registration-dir\") pod \"aws-ebs-csi-driver-node-qm42p\" (UID: \"6940cbba-826b-4e63-97cd-882dc35fdc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qm42p"
Sep 15 20:04:48.133129 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133020    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6940cbba-826b-4e63-97cd-882dc35fdc9c-registration-dir\") pod \"aws-ebs-csi-driver-node-qm42p\" (UID: \"6940cbba-826b-4e63-97cd-882dc35fdc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qm42p"
Sep 15 20:04:48.133129 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133025    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-host-var-lib-kubelet\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.133129 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133025    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-host-var-lib-cni-multus\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.133129 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133050    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6c3054ad-31af-4d06-9809-1eab5e2b140c-kubelet-config\") pod \"global-pull-secret-syncer-7hmwc\" (UID: \"6c3054ad-31af-4d06-9809-1eab5e2b140c\") " pod="kube-system/global-pull-secret-syncer-7hmwc"
Sep 15 20:04:48.133129 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133078    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-multus-cni-dir\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.133129 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133093    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6c3054ad-31af-4d06-9809-1eab5e2b140c-kubelet-config\") pod \"global-pull-secret-syncer-7hmwc\" (UID: \"6c3054ad-31af-4d06-9809-1eab5e2b140c\") " pod="kube-system/global-pull-secret-syncer-7hmwc"
Sep 15 20:04:48.133129 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133092    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-host-run-multus-certs\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.133129 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133125    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-host-run-multus-certs\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.133567 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133135    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-hostroot\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.133567 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133157    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6940cbba-826b-4e63-97cd-882dc35fdc9c-device-dir\") pod \"aws-ebs-csi-driver-node-qm42p\" (UID: \"6940cbba-826b-4e63-97cd-882dc35fdc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qm42p"
Sep 15 20:04:48.133567 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133178    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/48d7dc57-6de6-4a72-8da3-156c90d7a8ff-host\") pod \"node-ca-qk9ld\" (UID: \"48d7dc57-6de6-4a72-8da3-156c90d7a8ff\") " pod="openshift-image-registry/node-ca-qk9ld"
Sep 15 20:04:48.133567 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133191    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6940cbba-826b-4e63-97cd-882dc35fdc9c-device-dir\") pod \"aws-ebs-csi-driver-node-qm42p\" (UID: \"6940cbba-826b-4e63-97cd-882dc35fdc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qm42p"
Sep 15 20:04:48.133567 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133197    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-hostroot\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.133567 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133214    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/48d7dc57-6de6-4a72-8da3-156c90d7a8ff-host\") pod \"node-ca-qk9ld\" (UID: \"48d7dc57-6de6-4a72-8da3-156c90d7a8ff\") " pod="openshift-image-registry/node-ca-qk9ld"
Sep 15 20:04:48.133567 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133194    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-host-run-netns\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.133567 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133219    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-host-run-netns\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.133567 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133268    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c3054ad-31af-4d06-9809-1eab5e2b140c-original-pull-secret\") pod \"global-pull-secret-syncer-7hmwc\" (UID: \"6c3054ad-31af-4d06-9809-1eab5e2b140c\") " pod="kube-system/global-pull-secret-syncer-7hmwc"
Sep 15 20:04:48.133567 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133301    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/48d7dc57-6de6-4a72-8da3-156c90d7a8ff-serviceca\") pod \"node-ca-qk9ld\" (UID: \"48d7dc57-6de6-4a72-8da3-156c90d7a8ff\") " pod="openshift-image-registry/node-ca-qk9ld"
Sep 15 20:04:48.133567 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133325    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-cnibin\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.133567 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133361    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5368fcd5-c6e8-4f29-b93c-338fdd857bad-cni-binary-copy\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.133567 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133379    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-host-run-k8s-cni-cncf-io\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.133567 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133379    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-cnibin\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.133567 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:48.133384    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 20:04:48.133567 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133415    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5368fcd5-c6e8-4f29-b93c-338fdd857bad-multus-daemon-config\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.133567 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:48.133441    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c3054ad-31af-4d06-9809-1eab5e2b140c-original-pull-secret podName:6c3054ad-31af-4d06-9809-1eab5e2b140c nodeName:}" failed. No retries permitted until 2026-09-15 20:04:48.633428157 +0000 UTC m=+3.121892615 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c3054ad-31af-4d06-9809-1eab5e2b140c-original-pull-secret") pod "global-pull-secret-syncer-7hmwc" (UID: "6c3054ad-31af-4d06-9809-1eab5e2b140c") : object "kube-system"/"original-pull-secret" not registered
Sep 15 20:04:48.133567 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133440    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-host-run-k8s-cni-cncf-io\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.134279 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133454    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gm5tm\" (UniqueName: \"kubernetes.io/projected/6940cbba-826b-4e63-97cd-882dc35fdc9c-kube-api-access-gm5tm\") pod \"aws-ebs-csi-driver-node-qm42p\" (UID: \"6940cbba-826b-4e63-97cd-882dc35fdc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qm42p"
Sep 15 20:04:48.134279 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133481    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-multus-conf-dir\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.134279 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133498    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6940cbba-826b-4e63-97cd-882dc35fdc9c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qm42p\" (UID: \"6940cbba-826b-4e63-97cd-882dc35fdc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qm42p"
Sep 15 20:04:48.134279 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133515    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6940cbba-826b-4e63-97cd-882dc35fdc9c-sys-fs\") pod \"aws-ebs-csi-driver-node-qm42p\" (UID: \"6940cbba-826b-4e63-97cd-882dc35fdc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qm42p"
Sep 15 20:04:48.134279 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133612    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6940cbba-826b-4e63-97cd-882dc35fdc9c-sys-fs\") pod \"aws-ebs-csi-driver-node-qm42p\" (UID: \"6940cbba-826b-4e63-97cd-882dc35fdc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qm42p"
Sep 15 20:04:48.134279 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133583    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6940cbba-826b-4e63-97cd-882dc35fdc9c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qm42p\" (UID: \"6940cbba-826b-4e63-97cd-882dc35fdc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qm42p"
Sep 15 20:04:48.134279 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133648    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6c3054ad-31af-4d06-9809-1eab5e2b140c-dbus\") pod \"global-pull-secret-syncer-7hmwc\" (UID: \"6c3054ad-31af-4d06-9809-1eab5e2b140c\") " pod="kube-system/global-pull-secret-syncer-7hmwc"
Sep 15 20:04:48.134279 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133677    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-system-cni-dir\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.134279 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133564    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-multus-conf-dir\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.134279 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133698    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np6m9\" (UniqueName: \"kubernetes.io/projected/5368fcd5-c6e8-4f29-b93c-338fdd857bad-kube-api-access-np6m9\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.134279 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133732    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6940cbba-826b-4e63-97cd-882dc35fdc9c-socket-dir\") pod \"aws-ebs-csi-driver-node-qm42p\" (UID: \"6940cbba-826b-4e63-97cd-882dc35fdc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qm42p"
Sep 15 20:04:48.134279 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133755    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6940cbba-826b-4e63-97cd-882dc35fdc9c-etc-selinux\") pod \"aws-ebs-csi-driver-node-qm42p\" (UID: \"6940cbba-826b-4e63-97cd-882dc35fdc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qm42p"
Sep 15 20:04:48.134279 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133760    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-system-cni-dir\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.134279 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133747    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6c3054ad-31af-4d06-9809-1eab5e2b140c-dbus\") pod \"global-pull-secret-syncer-7hmwc\" (UID: \"6c3054ad-31af-4d06-9809-1eab5e2b140c\") " pod="kube-system/global-pull-secret-syncer-7hmwc"
Sep 15 20:04:48.134279 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133802    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-host-var-lib-cni-bin\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.134279 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133835    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c98sm\" (UniqueName: \"kubernetes.io/projected/48d7dc57-6de6-4a72-8da3-156c90d7a8ff-kube-api-access-c98sm\") pod \"node-ca-qk9ld\" (UID: \"48d7dc57-6de6-4a72-8da3-156c90d7a8ff\") " pod="openshift-image-registry/node-ca-qk9ld"
Sep 15 20:04:48.134279 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133857    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-multus-socket-dir-parent\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.134738 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133867    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/48d7dc57-6de6-4a72-8da3-156c90d7a8ff-serviceca\") pod \"node-ca-qk9ld\" (UID: \"48d7dc57-6de6-4a72-8da3-156c90d7a8ff\") " pod="openshift-image-registry/node-ca-qk9ld"
Sep 15 20:04:48.134738 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133881    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-etc-kubernetes\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.134738 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133885    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6940cbba-826b-4e63-97cd-882dc35fdc9c-socket-dir\") pod \"aws-ebs-csi-driver-node-qm42p\" (UID: \"6940cbba-826b-4e63-97cd-882dc35fdc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qm42p"
Sep 15 20:04:48.134738 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133858    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6940cbba-826b-4e63-97cd-882dc35fdc9c-etc-selinux\") pod \"aws-ebs-csi-driver-node-qm42p\" (UID: \"6940cbba-826b-4e63-97cd-882dc35fdc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qm42p"
Sep 15 20:04:48.134738 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133974    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-multus-socket-dir-parent\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.134738 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.133975    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-etc-kubernetes\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.134738 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.134004    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5368fcd5-c6e8-4f29-b93c-338fdd857bad-host-var-lib-cni-bin\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.134738 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.134025    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5368fcd5-c6e8-4f29-b93c-338fdd857bad-cni-binary-copy\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.134738 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.134059    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5368fcd5-c6e8-4f29-b93c-338fdd857bad-multus-daemon-config\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.141513 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.141433    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-155.ec2.internal" event={"ID":"6cec6a6594652885b2e8d4cf7af01cc1","Type":"ContainerStarted","Data":"1ca2b9ae04fe1ec86c2faa917d95d5308f1f6b89726668ff53f0c9e46173e9b5"}
Sep 15 20:04:48.142776 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.142749    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-155.ec2.internal" event={"ID":"207cad37faaa53a9bfb12816947f0fd7","Type":"ContainerStarted","Data":"36cd4d5896729bb6d68e4e7b41b362f02728f9bebfaf416594fc18175e7bd614"}
Sep 15 20:04:48.142879 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.142862    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gm5tm\" (UniqueName: \"kubernetes.io/projected/6940cbba-826b-4e63-97cd-882dc35fdc9c-kube-api-access-gm5tm\") pod \"aws-ebs-csi-driver-node-qm42p\" (UID: \"6940cbba-826b-4e63-97cd-882dc35fdc9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qm42p"
Sep 15 20:04:48.143193 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.143176    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c98sm\" (UniqueName: \"kubernetes.io/projected/48d7dc57-6de6-4a72-8da3-156c90d7a8ff-kube-api-access-c98sm\") pod \"node-ca-qk9ld\" (UID: \"48d7dc57-6de6-4a72-8da3-156c90d7a8ff\") " pod="openshift-image-registry/node-ca-qk9ld"
Sep 15 20:04:48.143262 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.143198    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-np6m9\" (UniqueName: \"kubernetes.io/projected/5368fcd5-c6e8-4f29-b93c-338fdd857bad-kube-api-access-np6m9\") pod \"multus-f7npj\" (UID: \"5368fcd5-c6e8-4f29-b93c-338fdd857bad\") " pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.209709 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.209685    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9r4qs"
Sep 15 20:04:48.217556 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.217528    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ldvcx"
Sep 15 20:04:48.218765 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:48.218740    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod794a8849_e693_4fd1_905d_addff4740cda.slice/crio-3afbeca4128ba3548489a77e3f56d7e75cdbc34edeb1a5ecf7385314dd835a74 WatchSource:0}: Error finding container 3afbeca4128ba3548489a77e3f56d7e75cdbc34edeb1a5ecf7385314dd835a74: Status 404 returned error can't find the container with id 3afbeca4128ba3548489a77e3f56d7e75cdbc34edeb1a5ecf7385314dd835a74
Sep 15 20:04:48.224147 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.224123    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-s8ddr"
Sep 15 20:04:48.225721 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:48.225701    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25927e9f_cdb1_49d8_8d63_ff66a3678f87.slice/crio-95c408a2505cbab000c889de99a3bb12937591eafbb7be1fa1b4d4cb82ed4d04 WatchSource:0}: Error finding container 95c408a2505cbab000c889de99a3bb12937591eafbb7be1fa1b4d4cb82ed4d04: Status 404 returned error can't find the container with id 95c408a2505cbab000c889de99a3bb12937591eafbb7be1fa1b4d4cb82ed4d04
Sep 15 20:04:48.229126 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.229111    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:04:48.232585 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:48.232565    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cd01117_433d_4273_a167_01344ad4f018.slice/crio-dea802ee3bc593952582953d02e3ce5de9a1af311e72a894a79a98ff1e248be7 WatchSource:0}: Error finding container dea802ee3bc593952582953d02e3ce5de9a1af311e72a894a79a98ff1e248be7: Status 404 returned error can't find the container with id dea802ee3bc593952582953d02e3ce5de9a1af311e72a894a79a98ff1e248be7
Sep 15 20:04:48.233505 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.233487    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4vdw2"
Sep 15 20:04:48.238033 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:48.238010    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bb16dab_3f3f_46a5_a0ba_3c19a7d24e5e.slice/crio-cc643ee3c13ed32eda8b58a7ae9c5b0d153f1e832706c1e6179cb701d9ac1fe6 WatchSource:0}: Error finding container cc643ee3c13ed32eda8b58a7ae9c5b0d153f1e832706c1e6179cb701d9ac1fe6: Status 404 returned error can't find the container with id cc643ee3c13ed32eda8b58a7ae9c5b0d153f1e832706c1e6179cb701d9ac1fe6
Sep 15 20:04:48.239456 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.239439    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rs6ws"
Sep 15 20:04:48.243201 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:48.243178    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6bec389_ca68_4845_8694_9571cf84fe3f.slice/crio-520381f73c1c375dac3b544397c9ec362014f07157dfd742d2cb13b991f24f34 WatchSource:0}: Error finding container 520381f73c1c375dac3b544397c9ec362014f07157dfd742d2cb13b991f24f34: Status 404 returned error can't find the container with id 520381f73c1c375dac3b544397c9ec362014f07157dfd742d2cb13b991f24f34
Sep 15 20:04:48.245039 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.245021    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qm42p"
Sep 15 20:04:48.250381 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:48.250336    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55164773_0e7d_48b1_b300_2f08b68ed675.slice/crio-7a874dcbed6acd3997588c36d5d8af9d7123be3f6210aa799b0b9bb427c5f4f4 WatchSource:0}: Error finding container 7a874dcbed6acd3997588c36d5d8af9d7123be3f6210aa799b0b9bb427c5f4f4: Status 404 returned error can't find the container with id 7a874dcbed6acd3997588c36d5d8af9d7123be3f6210aa799b0b9bb427c5f4f4
Sep 15 20:04:48.250662 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.250528    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qk9ld"
Sep 15 20:04:48.254834 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.254698    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-f7npj"
Sep 15 20:04:48.255007 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:48.254987    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6940cbba_826b_4e63_97cd_882dc35fdc9c.slice/crio-49fadf6901bbc96acfebb5f11f528ecd4519f80d9a29bb7b407b795f41c496ae WatchSource:0}: Error finding container 49fadf6901bbc96acfebb5f11f528ecd4519f80d9a29bb7b407b795f41c496ae: Status 404 returned error can't find the container with id 49fadf6901bbc96acfebb5f11f528ecd4519f80d9a29bb7b407b795f41c496ae
Sep 15 20:04:48.266294 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:04:48.266273    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5368fcd5_c6e8_4f29_b93c_338fdd857bad.slice/crio-3111de9441319158d0e55bd4c73bc7636c3de02831c9e70fecd20666ec2d9d03 WatchSource:0}: Error finding container 3111de9441319158d0e55bd4c73bc7636c3de02831c9e70fecd20666ec2d9d03: Status 404 returned error can't find the container with id 3111de9441319158d0e55bd4c73bc7636c3de02831c9e70fecd20666ec2d9d03
Sep 15 20:04:48.537773 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.537691    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-metrics-certs\") pod \"network-metrics-daemon-59mfs\" (UID: \"3b8d9342-6574-4d0d-a326-0299eb3b4c5d\") " pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:04:48.537946 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:48.537882    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:04:48.538018 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:48.537965    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-metrics-certs podName:3b8d9342-6574-4d0d-a326-0299eb3b4c5d nodeName:}" failed. No retries permitted until 2026-09-15 20:04:49.53794623 +0000 UTC m=+4.026410686 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-metrics-certs") pod "network-metrics-daemon-59mfs" (UID: "3b8d9342-6574-4d0d-a326-0299eb3b4c5d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:04:48.639029 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.638993    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sblk8\" (UniqueName: \"kubernetes.io/projected/43a5c998-df74-4a93-b02f-3d53c8be805a-kube-api-access-sblk8\") pod \"network-check-target-gzw9m\" (UID: \"43a5c998-df74-4a93-b02f-3d53c8be805a\") " pod="openshift-network-diagnostics/network-check-target-gzw9m"
Sep 15 20:04:48.639192 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.639083    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c3054ad-31af-4d06-9809-1eab5e2b140c-original-pull-secret\") pod \"global-pull-secret-syncer-7hmwc\" (UID: \"6c3054ad-31af-4d06-9809-1eab5e2b140c\") " pod="kube-system/global-pull-secret-syncer-7hmwc"
Sep 15 20:04:48.639255 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:48.639188    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 20:04:48.639255 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:48.639251    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c3054ad-31af-4d06-9809-1eab5e2b140c-original-pull-secret podName:6c3054ad-31af-4d06-9809-1eab5e2b140c nodeName:}" failed. No retries permitted until 2026-09-15 20:04:49.639237444 +0000 UTC m=+4.127701899 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c3054ad-31af-4d06-9809-1eab5e2b140c-original-pull-secret") pod "global-pull-secret-syncer-7hmwc" (UID: "6c3054ad-31af-4d06-9809-1eab5e2b140c") : object "kube-system"/"original-pull-secret" not registered
Sep 15 20:04:48.639354 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:48.639191    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 20:04:48.639354 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:48.639317    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 20:04:48.639354 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:48.639338    2576 projected.go:196] Error preparing data for projected volume kube-api-access-sblk8 for pod openshift-network-diagnostics/network-check-target-gzw9m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:04:48.639472 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:48.639395    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43a5c998-df74-4a93-b02f-3d53c8be805a-kube-api-access-sblk8 podName:43a5c998-df74-4a93-b02f-3d53c8be805a nodeName:}" failed. No retries permitted until 2026-09-15 20:04:49.639377812 +0000 UTC m=+4.127842280 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-sblk8" (UniqueName: "kubernetes.io/projected/43a5c998-df74-4a93-b02f-3d53c8be805a-kube-api-access-sblk8") pod "network-check-target-gzw9m" (UID: "43a5c998-df74-4a93-b02f-3d53c8be805a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:04:48.952908 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.952801    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-14 19:59:47 +0000 UTC" deadline="2028-06-13 12:17:16.11061783 +0000 UTC"
Sep 15 20:04:48.952908 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:48.952856    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15280h12m27.157766181s"
Sep 15 20:04:49.137557 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:49.137524    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7hmwc"
Sep 15 20:04:49.137718 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:49.137660    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7hmwc" podUID="6c3054ad-31af-4d06-9809-1eab5e2b140c"
Sep 15 20:04:49.147206 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:49.147176    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qk9ld" event={"ID":"48d7dc57-6de6-4a72-8da3-156c90d7a8ff","Type":"ContainerStarted","Data":"22b82d66abd52fa76e45d435091f77b178ce2297f5f5b68bcea492c5eaa3ae56"}
Sep 15 20:04:49.149840 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:49.149810    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rs6ws" event={"ID":"55164773-0e7d-48b1-b300-2f08b68ed675","Type":"ContainerStarted","Data":"7a874dcbed6acd3997588c36d5d8af9d7123be3f6210aa799b0b9bb427c5f4f4"}
Sep 15 20:04:49.156477 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:49.156428    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6st24" event={"ID":"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e","Type":"ContainerStarted","Data":"cc643ee3c13ed32eda8b58a7ae9c5b0d153f1e832706c1e6179cb701d9ac1fe6"}
Sep 15 20:04:49.158957 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:49.158933    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-s8ddr" event={"ID":"6cd01117-433d-4273-a167-01344ad4f018","Type":"ContainerStarted","Data":"dea802ee3bc593952582953d02e3ce5de9a1af311e72a894a79a98ff1e248be7"}
Sep 15 20:04:49.160420 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:49.160396    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9r4qs" event={"ID":"794a8849-e693-4fd1-905d-addff4740cda","Type":"ContainerStarted","Data":"3afbeca4128ba3548489a77e3f56d7e75cdbc34edeb1a5ecf7385314dd835a74"}
Sep 15 20:04:49.162576 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:49.162532    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-f7npj" event={"ID":"5368fcd5-c6e8-4f29-b93c-338fdd857bad","Type":"ContainerStarted","Data":"3111de9441319158d0e55bd4c73bc7636c3de02831c9e70fecd20666ec2d9d03"}
Sep 15 20:04:49.164249 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:49.164204    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qm42p" event={"ID":"6940cbba-826b-4e63-97cd-882dc35fdc9c","Type":"ContainerStarted","Data":"49fadf6901bbc96acfebb5f11f528ecd4519f80d9a29bb7b407b795f41c496ae"}
Sep 15 20:04:49.166125 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:49.166101    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4vdw2" event={"ID":"e6bec389-ca68-4845-8694-9571cf84fe3f","Type":"ContainerStarted","Data":"520381f73c1c375dac3b544397c9ec362014f07157dfd742d2cb13b991f24f34"}
Sep 15 20:04:49.168111 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:49.168085    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ldvcx" event={"ID":"25927e9f-cdb1-49d8-8d63-ff66a3678f87","Type":"ContainerStarted","Data":"95c408a2505cbab000c889de99a3bb12937591eafbb7be1fa1b4d4cb82ed4d04"}
Sep 15 20:04:49.547185 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:49.547048    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-metrics-certs\") pod \"network-metrics-daemon-59mfs\" (UID: \"3b8d9342-6574-4d0d-a326-0299eb3b4c5d\") " pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:04:49.547373 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:49.547218    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:04:49.547373 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:49.547301    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-metrics-certs podName:3b8d9342-6574-4d0d-a326-0299eb3b4c5d nodeName:}" failed. No retries permitted until 2026-09-15 20:04:51.547278956 +0000 UTC m=+6.035743414 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-metrics-certs") pod "network-metrics-daemon-59mfs" (UID: "3b8d9342-6574-4d0d-a326-0299eb3b4c5d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:04:49.648197 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:49.648157    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c3054ad-31af-4d06-9809-1eab5e2b140c-original-pull-secret\") pod \"global-pull-secret-syncer-7hmwc\" (UID: \"6c3054ad-31af-4d06-9809-1eab5e2b140c\") " pod="kube-system/global-pull-secret-syncer-7hmwc"
Sep 15 20:04:49.648356 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:49.648215    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sblk8\" (UniqueName: \"kubernetes.io/projected/43a5c998-df74-4a93-b02f-3d53c8be805a-kube-api-access-sblk8\") pod \"network-check-target-gzw9m\" (UID: \"43a5c998-df74-4a93-b02f-3d53c8be805a\") " pod="openshift-network-diagnostics/network-check-target-gzw9m"
Sep 15 20:04:49.648356 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:49.648328    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 20:04:49.648488 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:49.648376    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 20:04:49.648488 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:49.648397    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 20:04:49.648488 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:49.648410    2576 projected.go:196] Error preparing data for projected volume kube-api-access-sblk8 for pod openshift-network-diagnostics/network-check-target-gzw9m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:04:49.648488 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:49.648410    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c3054ad-31af-4d06-9809-1eab5e2b140c-original-pull-secret podName:6c3054ad-31af-4d06-9809-1eab5e2b140c nodeName:}" failed. No retries permitted until 2026-09-15 20:04:51.648389743 +0000 UTC m=+6.136854216 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c3054ad-31af-4d06-9809-1eab5e2b140c-original-pull-secret") pod "global-pull-secret-syncer-7hmwc" (UID: "6c3054ad-31af-4d06-9809-1eab5e2b140c") : object "kube-system"/"original-pull-secret" not registered
Sep 15 20:04:49.648488 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:49.648468    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43a5c998-df74-4a93-b02f-3d53c8be805a-kube-api-access-sblk8 podName:43a5c998-df74-4a93-b02f-3d53c8be805a nodeName:}" failed. No retries permitted until 2026-09-15 20:04:51.648453243 +0000 UTC m=+6.136917704 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-sblk8" (UniqueName: "kubernetes.io/projected/43a5c998-df74-4a93-b02f-3d53c8be805a-kube-api-access-sblk8") pod "network-check-target-gzw9m" (UID: "43a5c998-df74-4a93-b02f-3d53c8be805a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:04:49.953908 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:49.953786    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-14 19:59:47 +0000 UTC" deadline="2028-05-18 21:24:31.257254266 +0000 UTC"
Sep 15 20:04:49.953908 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:49.953827    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14665h19m41.303432001s"
Sep 15 20:04:50.140182 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:50.140148    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzw9m"
Sep 15 20:04:50.140323 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:50.140272    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gzw9m" podUID="43a5c998-df74-4a93-b02f-3d53c8be805a"
Sep 15 20:04:50.141119 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:50.141097    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:04:50.141233 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:50.141213    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59mfs" podUID="3b8d9342-6574-4d0d-a326-0299eb3b4c5d"
Sep 15 20:04:51.137975 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:51.137442    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7hmwc"
Sep 15 20:04:51.137975 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:51.137578    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7hmwc" podUID="6c3054ad-31af-4d06-9809-1eab5e2b140c"
Sep 15 20:04:51.562671 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:51.562593    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-metrics-certs\") pod \"network-metrics-daemon-59mfs\" (UID: \"3b8d9342-6574-4d0d-a326-0299eb3b4c5d\") " pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:04:51.562838 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:51.562779    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:04:51.562897 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:51.562840    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-metrics-certs podName:3b8d9342-6574-4d0d-a326-0299eb3b4c5d nodeName:}" failed. No retries permitted until 2026-09-15 20:04:55.562820469 +0000 UTC m=+10.051284928 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-metrics-certs") pod "network-metrics-daemon-59mfs" (UID: "3b8d9342-6574-4d0d-a326-0299eb3b4c5d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:04:51.664070 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:51.663265    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c3054ad-31af-4d06-9809-1eab5e2b140c-original-pull-secret\") pod \"global-pull-secret-syncer-7hmwc\" (UID: \"6c3054ad-31af-4d06-9809-1eab5e2b140c\") " pod="kube-system/global-pull-secret-syncer-7hmwc"
Sep 15 20:04:51.664070 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:51.663321    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sblk8\" (UniqueName: \"kubernetes.io/projected/43a5c998-df74-4a93-b02f-3d53c8be805a-kube-api-access-sblk8\") pod \"network-check-target-gzw9m\" (UID: \"43a5c998-df74-4a93-b02f-3d53c8be805a\") " pod="openshift-network-diagnostics/network-check-target-gzw9m"
Sep 15 20:04:51.664070 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:51.663505    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 20:04:51.664070 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:51.663528    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 20:04:51.664070 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:51.663541    2576 projected.go:196] Error preparing data for projected volume kube-api-access-sblk8 for pod openshift-network-diagnostics/network-check-target-gzw9m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:04:51.664070 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:51.663607    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43a5c998-df74-4a93-b02f-3d53c8be805a-kube-api-access-sblk8 podName:43a5c998-df74-4a93-b02f-3d53c8be805a nodeName:}" failed. No retries permitted until 2026-09-15 20:04:55.663580619 +0000 UTC m=+10.152045091 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-sblk8" (UniqueName: "kubernetes.io/projected/43a5c998-df74-4a93-b02f-3d53c8be805a-kube-api-access-sblk8") pod "network-check-target-gzw9m" (UID: "43a5c998-df74-4a93-b02f-3d53c8be805a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:04:51.664070 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:51.664003    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 20:04:51.664070 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:51.664041    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c3054ad-31af-4d06-9809-1eab5e2b140c-original-pull-secret podName:6c3054ad-31af-4d06-9809-1eab5e2b140c nodeName:}" failed. No retries permitted until 2026-09-15 20:04:55.664030774 +0000 UTC m=+10.152495228 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c3054ad-31af-4d06-9809-1eab5e2b140c-original-pull-secret") pod "global-pull-secret-syncer-7hmwc" (UID: "6c3054ad-31af-4d06-9809-1eab5e2b140c") : object "kube-system"/"original-pull-secret" not registered
Sep 15 20:04:52.138000 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:52.137968    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzw9m"
Sep 15 20:04:52.138443 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:52.138101    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gzw9m" podUID="43a5c998-df74-4a93-b02f-3d53c8be805a"
Sep 15 20:04:52.138443 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:52.138236    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:04:52.138443 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:52.138355    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59mfs" podUID="3b8d9342-6574-4d0d-a326-0299eb3b4c5d"
Sep 15 20:04:53.137577 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:53.137537    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7hmwc"
Sep 15 20:04:53.137751 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:53.137675    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7hmwc" podUID="6c3054ad-31af-4d06-9809-1eab5e2b140c"
Sep 15 20:04:54.137064 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:54.137032    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:04:54.137064 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:54.137055    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzw9m"
Sep 15 20:04:54.137498 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:54.137152    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gzw9m" podUID="43a5c998-df74-4a93-b02f-3d53c8be805a"
Sep 15 20:04:54.137498 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:54.137277    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59mfs" podUID="3b8d9342-6574-4d0d-a326-0299eb3b4c5d"
Sep 15 20:04:55.137707 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:55.137674    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7hmwc"
Sep 15 20:04:55.138148 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:55.137804    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7hmwc" podUID="6c3054ad-31af-4d06-9809-1eab5e2b140c"
Sep 15 20:04:55.598995 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:55.598839    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-metrics-certs\") pod \"network-metrics-daemon-59mfs\" (UID: \"3b8d9342-6574-4d0d-a326-0299eb3b4c5d\") " pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:04:55.599655 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:55.599252    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:04:55.599655 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:55.599324    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-metrics-certs podName:3b8d9342-6574-4d0d-a326-0299eb3b4c5d nodeName:}" failed. No retries permitted until 2026-09-15 20:05:03.599303537 +0000 UTC m=+18.087768000 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-metrics-certs") pod "network-metrics-daemon-59mfs" (UID: "3b8d9342-6574-4d0d-a326-0299eb3b4c5d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:04:55.699900 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:55.699150    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c3054ad-31af-4d06-9809-1eab5e2b140c-original-pull-secret\") pod \"global-pull-secret-syncer-7hmwc\" (UID: \"6c3054ad-31af-4d06-9809-1eab5e2b140c\") " pod="kube-system/global-pull-secret-syncer-7hmwc"
Sep 15 20:04:55.699900 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:55.699204    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sblk8\" (UniqueName: \"kubernetes.io/projected/43a5c998-df74-4a93-b02f-3d53c8be805a-kube-api-access-sblk8\") pod \"network-check-target-gzw9m\" (UID: \"43a5c998-df74-4a93-b02f-3d53c8be805a\") " pod="openshift-network-diagnostics/network-check-target-gzw9m"
Sep 15 20:04:55.699900 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:55.699342    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 20:04:55.699900 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:55.699361    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 20:04:55.699900 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:55.699373    2576 projected.go:196] Error preparing data for projected volume kube-api-access-sblk8 for pod openshift-network-diagnostics/network-check-target-gzw9m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:04:55.699900 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:55.699437    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43a5c998-df74-4a93-b02f-3d53c8be805a-kube-api-access-sblk8 podName:43a5c998-df74-4a93-b02f-3d53c8be805a nodeName:}" failed. No retries permitted until 2026-09-15 20:05:03.69941953 +0000 UTC m=+18.187884007 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-sblk8" (UniqueName: "kubernetes.io/projected/43a5c998-df74-4a93-b02f-3d53c8be805a-kube-api-access-sblk8") pod "network-check-target-gzw9m" (UID: "43a5c998-df74-4a93-b02f-3d53c8be805a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:04:55.699900 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:55.699817    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 20:04:55.699900 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:55.699865    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c3054ad-31af-4d06-9809-1eab5e2b140c-original-pull-secret podName:6c3054ad-31af-4d06-9809-1eab5e2b140c nodeName:}" failed. No retries permitted until 2026-09-15 20:05:03.699849625 +0000 UTC m=+18.188314086 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c3054ad-31af-4d06-9809-1eab5e2b140c-original-pull-secret") pod "global-pull-secret-syncer-7hmwc" (UID: "6c3054ad-31af-4d06-9809-1eab5e2b140c") : object "kube-system"/"original-pull-secret" not registered
Sep 15 20:04:56.140664 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:56.140635    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:04:56.141122 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:56.140748    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59mfs" podUID="3b8d9342-6574-4d0d-a326-0299eb3b4c5d"
Sep 15 20:04:56.141122 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:56.140787    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzw9m"
Sep 15 20:04:56.141122 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:56.140825    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gzw9m" podUID="43a5c998-df74-4a93-b02f-3d53c8be805a"
Sep 15 20:04:57.136983 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:57.136950    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7hmwc"
Sep 15 20:04:57.137145 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:57.137056    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7hmwc" podUID="6c3054ad-31af-4d06-9809-1eab5e2b140c"
Sep 15 20:04:58.137099 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:58.137028    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:04:58.137527 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:58.137027    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzw9m"
Sep 15 20:04:58.137527 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:58.137182    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59mfs" podUID="3b8d9342-6574-4d0d-a326-0299eb3b4c5d"
Sep 15 20:04:58.137527 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:58.137302    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gzw9m" podUID="43a5c998-df74-4a93-b02f-3d53c8be805a"
Sep 15 20:04:59.137319 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:04:59.137287    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7hmwc"
Sep 15 20:04:59.137763 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:04:59.137392    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7hmwc" podUID="6c3054ad-31af-4d06-9809-1eab5e2b140c"
Sep 15 20:05:00.137521 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:00.137480    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:05:00.137990 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:00.137632    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59mfs" podUID="3b8d9342-6574-4d0d-a326-0299eb3b4c5d"
Sep 15 20:05:00.137990 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:00.137688    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzw9m"
Sep 15 20:05:00.137990 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:00.137791    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gzw9m" podUID="43a5c998-df74-4a93-b02f-3d53c8be805a"
Sep 15 20:05:01.137875 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:01.137836    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7hmwc"
Sep 15 20:05:01.138290 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:01.137994    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7hmwc" podUID="6c3054ad-31af-4d06-9809-1eab5e2b140c"
Sep 15 20:05:02.137071 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:02.137041    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:05:02.137203 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:02.137151    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59mfs" podUID="3b8d9342-6574-4d0d-a326-0299eb3b4c5d"
Sep 15 20:05:02.137240 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:02.137206    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzw9m"
Sep 15 20:05:02.137303 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:02.137286    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gzw9m" podUID="43a5c998-df74-4a93-b02f-3d53c8be805a"
Sep 15 20:05:03.137789 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:03.137756    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7hmwc"
Sep 15 20:05:03.138306 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:03.137887    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7hmwc" podUID="6c3054ad-31af-4d06-9809-1eab5e2b140c"
Sep 15 20:05:03.657650 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:03.657610    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-metrics-certs\") pod \"network-metrics-daemon-59mfs\" (UID: \"3b8d9342-6574-4d0d-a326-0299eb3b4c5d\") " pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:05:03.657825 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:03.657789    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:05:03.657896 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:03.657875    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-metrics-certs podName:3b8d9342-6574-4d0d-a326-0299eb3b4c5d nodeName:}" failed. No retries permitted until 2026-09-15 20:05:19.657852754 +0000 UTC m=+34.146317213 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-metrics-certs") pod "network-metrics-daemon-59mfs" (UID: "3b8d9342-6574-4d0d-a326-0299eb3b4c5d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:05:03.758171 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:03.758136    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sblk8\" (UniqueName: \"kubernetes.io/projected/43a5c998-df74-4a93-b02f-3d53c8be805a-kube-api-access-sblk8\") pod \"network-check-target-gzw9m\" (UID: \"43a5c998-df74-4a93-b02f-3d53c8be805a\") " pod="openshift-network-diagnostics/network-check-target-gzw9m"
Sep 15 20:05:03.758338 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:03.758220    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c3054ad-31af-4d06-9809-1eab5e2b140c-original-pull-secret\") pod \"global-pull-secret-syncer-7hmwc\" (UID: \"6c3054ad-31af-4d06-9809-1eab5e2b140c\") " pod="kube-system/global-pull-secret-syncer-7hmwc"
Sep 15 20:05:03.758338 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:03.758310    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 20:05:03.758441 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:03.758360    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c3054ad-31af-4d06-9809-1eab5e2b140c-original-pull-secret podName:6c3054ad-31af-4d06-9809-1eab5e2b140c nodeName:}" failed. No retries permitted until 2026-09-15 20:05:19.758346309 +0000 UTC m=+34.246810763 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c3054ad-31af-4d06-9809-1eab5e2b140c-original-pull-secret") pod "global-pull-secret-syncer-7hmwc" (UID: "6c3054ad-31af-4d06-9809-1eab5e2b140c") : object "kube-system"/"original-pull-secret" not registered
Sep 15 20:05:03.758441 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:03.758312    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 20:05:03.758441 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:03.758407    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 20:05:03.758441 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:03.758421    2576 projected.go:196] Error preparing data for projected volume kube-api-access-sblk8 for pod openshift-network-diagnostics/network-check-target-gzw9m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:05:03.758587 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:03.758464    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43a5c998-df74-4a93-b02f-3d53c8be805a-kube-api-access-sblk8 podName:43a5c998-df74-4a93-b02f-3d53c8be805a nodeName:}" failed. No retries permitted until 2026-09-15 20:05:19.758453097 +0000 UTC m=+34.246917564 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-sblk8" (UniqueName: "kubernetes.io/projected/43a5c998-df74-4a93-b02f-3d53c8be805a-kube-api-access-sblk8") pod "network-check-target-gzw9m" (UID: "43a5c998-df74-4a93-b02f-3d53c8be805a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:05:04.137130 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:04.137048    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:05:04.137271 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:04.137200    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59mfs" podUID="3b8d9342-6574-4d0d-a326-0299eb3b4c5d"
Sep 15 20:05:04.137271 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:04.137255    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzw9m"
Sep 15 20:05:04.137368 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:04.137341    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gzw9m" podUID="43a5c998-df74-4a93-b02f-3d53c8be805a"
Sep 15 20:05:05.137183 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:05.137138    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7hmwc"
Sep 15 20:05:05.137600 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:05.137279    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7hmwc" podUID="6c3054ad-31af-4d06-9809-1eab5e2b140c"
Sep 15 20:05:06.138763 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:06.138666    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzw9m"
Sep 15 20:05:06.139122 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:06.138779    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:05:06.139122 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:06.138795    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gzw9m" podUID="43a5c998-df74-4a93-b02f-3d53c8be805a"
Sep 15 20:05:06.139122 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:06.139091    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59mfs" podUID="3b8d9342-6574-4d0d-a326-0299eb3b4c5d"
Sep 15 20:05:06.207038 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:06.207010    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-f7npj" event={"ID":"5368fcd5-c6e8-4f29-b93c-338fdd857bad","Type":"ContainerStarted","Data":"ea7605b04f591f4efac3494d9fbd4d05309346d63502bb0e718ba6cbfd1cb6ba"}
Sep 15 20:05:06.208737 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:06.208524    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ldvcx" event={"ID":"25927e9f-cdb1-49d8-8d63-ff66a3678f87","Type":"ContainerStarted","Data":"294881e2efa17507aca237c7870f5e9d10db62e2661b85b8d97ed8228d5ae943"}
Sep 15 20:05:06.210683 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:06.210663    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-155.ec2.internal" event={"ID":"6cec6a6594652885b2e8d4cf7af01cc1","Type":"ContainerStarted","Data":"d1b514316c257135dc83d7bdc725c056e0f249522a88ea758c73318b84503ae8"}
Sep 15 20:05:06.233263 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:06.233208    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-f7npj" podStartSLOduration=2.678579991 podStartE2EDuration="20.233192726s" totalImagesPullingTime="17.554612735s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:05:06.232131051 +0000 UTC m=+20.720595527" watchObservedRunningTime="2026-09-15 20:05:06.233192726 +0000 UTC m=+20.721657204"
Sep 15 20:05:06.299548 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:06.299501    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-155.ec2.internal" podStartSLOduration=19.299486352 podStartE2EDuration="19.299486352s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:04:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:05:06.261001995 +0000 UTC m=+20.749466471" watchObservedRunningTime="2026-09-15 20:05:06.299486352 +0000 UTC m=+20.787950810"
Sep 15 20:05:06.299695 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:06.299665    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ldvcx" podStartSLOduration=2.902213187 podStartE2EDuration="20.299658165s" totalImagesPullingTime="17.397444978s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:05:06.299617709 +0000 UTC m=+20.788082184" watchObservedRunningTime="2026-09-15 20:05:06.299658165 +0000 UTC m=+20.788122641"
Sep 15 20:05:07.137931 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:07.137747    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7hmwc"
Sep 15 20:05:07.138076 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:07.138043    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7hmwc" podUID="6c3054ad-31af-4d06-9809-1eab5e2b140c"
Sep 15 20:05:07.217865 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:07.217821    2576 generic.go:356] "Generic (PLEG): container finished" podID="207cad37faaa53a9bfb12816947f0fd7" containerID="0d8ed3dc677fa60c0a40cb0eebc6582717cc2770b12299d89b3e7ad00dbc21d6" exitCode=0
Sep 15 20:05:07.218717 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:07.217914    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-155.ec2.internal" event={"ID":"207cad37faaa53a9bfb12816947f0fd7","Type":"ContainerDied","Data":"0d8ed3dc677fa60c0a40cb0eebc6582717cc2770b12299d89b3e7ad00dbc21d6"}
Sep 15 20:05:07.221454 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:07.221423    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qm42p" event={"ID":"6940cbba-826b-4e63-97cd-882dc35fdc9c","Type":"ContainerStarted","Data":"b8a1eef896bcb46857c25b2e44c2a91f4d6c41f1f425f68ef017d8b4ce9003a5"}
Sep 15 20:05:07.223270 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:07.223218    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4vdw2" event={"ID":"e6bec389-ca68-4845-8694-9571cf84fe3f","Type":"ContainerStarted","Data":"e720cf8a2c1dda930af5d701c76df00e621461b9959a0c4bf3e7c7aeee2b58dc"}
Sep 15 20:05:07.224916 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:07.224891    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qk9ld" event={"ID":"48d7dc57-6de6-4a72-8da3-156c90d7a8ff","Type":"ContainerStarted","Data":"98430955292a62d7a3e193e147318d333a53b180d151f5600c118501a02b4415"}
Sep 15 20:05:07.226574 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:07.226549    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rs6ws" event={"ID":"55164773-0e7d-48b1-b300-2f08b68ed675","Type":"ContainerStarted","Data":"0a65efb77ced50fa0135508191c15a749ed7de6deef0d595cb8c6dec49e52fee"}
Sep 15 20:05:07.230457 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:07.230435    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6st24" event={"ID":"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e","Type":"ContainerStarted","Data":"870bbbff28a4c93be1b45bcc7e5f0553c050a0018fd4c998ec54395782a05f21"}
Sep 15 20:05:07.230553 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:07.230511    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6st24" event={"ID":"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e","Type":"ContainerStarted","Data":"d11ce0732158ea66de8f0fb38ca8ba03857a4cf73d081526b92fbcd03c2e55eb"}
Sep 15 20:05:07.230620 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:07.230564    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6st24" event={"ID":"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e","Type":"ContainerStarted","Data":"a9860ef85e7c96c8b18ea77eabfe9a353ccceeb3a4d1873de0c9447a421d3d79"}
Sep 15 20:05:07.230620 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:07.230610    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6st24" event={"ID":"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e","Type":"ContainerStarted","Data":"70f47e8ae96e30f3133ff9906ef97e94c67cac4619cf15de6f256a7852bdd708"}
Sep 15 20:05:07.230726 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:07.230658    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6st24" event={"ID":"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e","Type":"ContainerStarted","Data":"6396d3adb166c4554eba212dc5a3260ec29968748a00ea05c946b9789e3b08cc"}
Sep 15 20:05:07.230726 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:07.230698    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6st24" event={"ID":"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e","Type":"ContainerStarted","Data":"4a20477b128810cdef8446a08310c4fa7b9c9056dde69466e713664f27a535d0"}
Sep 15 20:05:07.232014 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:07.231993    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-s8ddr" event={"ID":"6cd01117-433d-4273-a167-01344ad4f018","Type":"ContainerStarted","Data":"eb7acfeb2b1ef39fe470ac001bae2d2028eb8444b48370e492fa013fea651946"}
Sep 15 20:05:07.233823 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:07.233801    2576 generic.go:356] "Generic (PLEG): container finished" podID="794a8849-e693-4fd1-905d-addff4740cda" containerID="c440e355fbd46208ff36357b1110f2253fbf61b70de643f09839d2a5ea7521f5" exitCode=0
Sep 15 20:05:07.233901 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:07.233876    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9r4qs" event={"ID":"794a8849-e693-4fd1-905d-addff4740cda","Type":"ContainerDied","Data":"c440e355fbd46208ff36357b1110f2253fbf61b70de643f09839d2a5ea7521f5"}
Sep 15 20:05:07.249801 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:07.249752    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qk9ld" podStartSLOduration=3.888191685 podStartE2EDuration="21.249733702s" totalImagesPullingTime="17.361542017s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:05:07.249338601 +0000 UTC m=+21.737803081" watchObservedRunningTime="2026-09-15 20:05:07.249733702 +0000 UTC m=+21.738198179"
Sep 15 20:05:07.263443 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:07.263386    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-s8ddr" podStartSLOduration=3.895440309 podStartE2EDuration="21.263367605s" totalImagesPullingTime="17.367927296s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:05:07.26328181 +0000 UTC m=+21.751746287" watchObservedRunningTime="2026-09-15 20:05:07.263367605 +0000 UTC m=+21.751832081"
Sep 15 20:05:07.299125 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:07.298975    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4vdw2" podStartSLOduration=11.12274121 podStartE2EDuration="21.298958466s" totalImagesPullingTime="10.176217256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:05:07.298590116 +0000 UTC m=+21.787054593" watchObservedRunningTime="2026-09-15 20:05:07.298958466 +0000 UTC m=+21.787422939"
Sep 15 20:05:07.316431 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:07.316362    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rs6ws" podStartSLOduration=3.94394184 podStartE2EDuration="21.31634716s" totalImagesPullingTime="17.37240532s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:05:07.316004832 +0000 UTC m=+21.804469311" watchObservedRunningTime="2026-09-15 20:05:07.31634716 +0000 UTC m=+21.804811635"
Sep 15 20:05:07.542570 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:07.542549    2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 15 20:05:07.998034 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:07.997896    2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-15T20:05:07.542565204Z","UUID":"f3dffe54-dde0-47ac-89a2-847becf9f83d","Handler":null,"Name":"","Endpoint":""}
Sep 15 20:05:07.999694 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:07.999672    2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 15 20:05:07.999820 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:07.999701    2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 15 20:05:08.032318 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:08.032291    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4vdw2"
Sep 15 20:05:08.032832 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:08.032815    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4vdw2"
Sep 15 20:05:08.137269 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:08.137242    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzw9m"
Sep 15 20:05:08.137269 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:08.137257    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:05:08.137491 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:08.137349    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gzw9m" podUID="43a5c998-df74-4a93-b02f-3d53c8be805a"
Sep 15 20:05:08.137555 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:08.137481    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59mfs" podUID="3b8d9342-6574-4d0d-a326-0299eb3b4c5d"
Sep 15 20:05:08.237645 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:08.237616    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-155.ec2.internal" event={"ID":"207cad37faaa53a9bfb12816947f0fd7","Type":"ContainerStarted","Data":"4d1d628eeb294558a2223af9e079492052a4701acaa869fca260363d99cd98c2"}
Sep 15 20:05:08.239497 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:08.239473    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qm42p" event={"ID":"6940cbba-826b-4e63-97cd-882dc35fdc9c","Type":"ContainerStarted","Data":"3397402dd0a3171ec59ae690be3aa7e418e4394a5ca710e5fd548307d3cd44d5"}
Sep 15 20:05:08.239690 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:08.239673    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4vdw2"
Sep 15 20:05:08.240182 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:08.240144    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4vdw2"
Sep 15 20:05:08.268770 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:08.268681    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-155.ec2.internal" podStartSLOduration=21.268668193 podStartE2EDuration="21.268668193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:04:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:05:08.25368405 +0000 UTC m=+22.742148528" watchObservedRunningTime="2026-09-15 20:05:08.268668193 +0000 UTC m=+22.757132650"
Sep 15 20:05:09.137316 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:09.137249    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7hmwc"
Sep 15 20:05:09.137467 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:09.137366    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7hmwc" podUID="6c3054ad-31af-4d06-9809-1eab5e2b140c"
Sep 15 20:05:09.245878 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:09.245839    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6st24" event={"ID":"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e","Type":"ContainerStarted","Data":"6237db9cf2c3feafa7d5b16f6b35bc086fcc4264f8c84417d8a6b28635a03a35"}
Sep 15 20:05:09.248204 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:09.248175    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qm42p" event={"ID":"6940cbba-826b-4e63-97cd-882dc35fdc9c","Type":"ContainerStarted","Data":"411a6b99e0ec2160a6fa6c774cc449b7c78ec75a6a4c1af8d3c4743b921555cc"}
Sep 15 20:05:09.265688 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:09.265639    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qm42p" podStartSLOduration=11.268870263 podStartE2EDuration="23.265621702s" totalImagesPullingTime="11.996751439s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:04:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:05:09.265152093 +0000 UTC m=+23.753616570" watchObservedRunningTime="2026-09-15 20:05:09.265621702 +0000 UTC m=+23.754086179"
Sep 15 20:05:10.137484 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:10.137458    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzw9m"
Sep 15 20:05:10.137764 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:10.137575    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gzw9m" podUID="43a5c998-df74-4a93-b02f-3d53c8be805a"
Sep 15 20:05:10.137764 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:10.137622    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:05:10.137764 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:10.137754    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59mfs" podUID="3b8d9342-6574-4d0d-a326-0299eb3b4c5d"
Sep 15 20:05:11.137657 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:11.137478    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7hmwc"
Sep 15 20:05:11.138113 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:11.137763    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7hmwc" podUID="6c3054ad-31af-4d06-9809-1eab5e2b140c"
Sep 15 20:05:11.257839 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:11.257760    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6st24" event={"ID":"6bb16dab-3f3f-46a5-a0ba-3c19a7d24e5e","Type":"ContainerStarted","Data":"62246b98e9b7cd787c87aa585004d611aef0b1dea150182919609501eb4eba6b"}
Sep 15 20:05:11.258247 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:11.258128    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:05:11.258247 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:11.258209    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:05:11.275512 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:11.275479    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:05:11.293651 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:11.293606    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6st24" podStartSLOduration=7.164553149 podStartE2EDuration="25.293588362s" totalImagesPullingTime="18.129035213s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:05:11.292956728 +0000 UTC m=+25.781421206" watchObservedRunningTime="2026-09-15 20:05:11.293588362 +0000 UTC m=+25.782052839"
Sep 15 20:05:12.136839 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:12.136803    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzw9m"
Sep 15 20:05:12.136839 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:12.136842    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:05:12.137170 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:12.136946    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gzw9m" podUID="43a5c998-df74-4a93-b02f-3d53c8be805a"
Sep 15 20:05:12.137170 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:12.137057    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59mfs" podUID="3b8d9342-6574-4d0d-a326-0299eb3b4c5d"
Sep 15 20:05:12.261523 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:12.261491    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:05:12.296744 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:12.296485    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:05:13.010838 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:13.010792    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7hmwc"]
Sep 15 20:05:13.011058 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:13.010948    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7hmwc"
Sep 15 20:05:13.011058 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:13.011048    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7hmwc" podUID="6c3054ad-31af-4d06-9809-1eab5e2b140c"
Sep 15 20:05:13.016152 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:13.016125    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gzw9m"]
Sep 15 20:05:13.016289 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:13.016247    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzw9m"
Sep 15 20:05:13.016378 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:13.016350    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gzw9m" podUID="43a5c998-df74-4a93-b02f-3d53c8be805a"
Sep 15 20:05:13.019223 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:13.019200    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-59mfs"]
Sep 15 20:05:13.019333 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:13.019291    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:05:13.019407 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:13.019387    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59mfs" podUID="3b8d9342-6574-4d0d-a326-0299eb3b4c5d"
Sep 15 20:05:14.266761 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:14.266728    2576 generic.go:356] "Generic (PLEG): container finished" podID="794a8849-e693-4fd1-905d-addff4740cda" containerID="983a5bf1bd396b9b02efb78ef29ad522205c7f5a9114299e9c3444dcf25841c8" exitCode=0
Sep 15 20:05:14.267489 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:14.266817    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9r4qs" event={"ID":"794a8849-e693-4fd1-905d-addff4740cda","Type":"ContainerDied","Data":"983a5bf1bd396b9b02efb78ef29ad522205c7f5a9114299e9c3444dcf25841c8"}
Sep 15 20:05:15.136856 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:15.136823    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzw9m"
Sep 15 20:05:15.136856 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:15.136844    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:05:15.137052 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:15.136870    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7hmwc"
Sep 15 20:05:15.137052 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:15.136963    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59mfs" podUID="3b8d9342-6574-4d0d-a326-0299eb3b4c5d"
Sep 15 20:05:15.137052 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:15.137000    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7hmwc" podUID="6c3054ad-31af-4d06-9809-1eab5e2b140c"
Sep 15 20:05:15.137167 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:15.137072    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gzw9m" podUID="43a5c998-df74-4a93-b02f-3d53c8be805a"
Sep 15 20:05:15.270041 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:15.270013    2576 generic.go:356] "Generic (PLEG): container finished" podID="794a8849-e693-4fd1-905d-addff4740cda" containerID="f2d874b9c3119a9580e27c7e701931dfeb60bf71f13c4204d2b234b7bf1c9a4f" exitCode=0
Sep 15 20:05:15.270492 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:15.270053    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9r4qs" event={"ID":"794a8849-e693-4fd1-905d-addff4740cda","Type":"ContainerDied","Data":"f2d874b9c3119a9580e27c7e701931dfeb60bf71f13c4204d2b234b7bf1c9a4f"}
Sep 15 20:05:16.275357 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:16.275046    2576 generic.go:356] "Generic (PLEG): container finished" podID="794a8849-e693-4fd1-905d-addff4740cda" containerID="1cc5896bee57c0dcbd3891fb2a323430984d1dfb8af10e0755502405c68c1bf8" exitCode=0
Sep 15 20:05:16.275357 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:16.275139    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9r4qs" event={"ID":"794a8849-e693-4fd1-905d-addff4740cda","Type":"ContainerDied","Data":"1cc5896bee57c0dcbd3891fb2a323430984d1dfb8af10e0755502405c68c1bf8"}
Sep 15 20:05:17.137515 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:17.137485    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:05:17.137515 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:17.137521    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7hmwc"
Sep 15 20:05:17.137735 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:17.137489    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzw9m"
Sep 15 20:05:17.137735 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:17.137622    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59mfs" podUID="3b8d9342-6574-4d0d-a326-0299eb3b4c5d"
Sep 15 20:05:17.137735 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:17.137683    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gzw9m" podUID="43a5c998-df74-4a93-b02f-3d53c8be805a"
Sep 15 20:05:17.137880 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:17.137741    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7hmwc" podUID="6c3054ad-31af-4d06-9809-1eab5e2b140c"
Sep 15 20:05:18.788185 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.788162    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-155.ec2.internal" event="NodeReady"
Sep 15 20:05:18.788564 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.788273    2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 15 20:05:18.819967 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.819938    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-2kszh"]
Sep 15 20:05:18.831147 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.831125    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-557757c5cd-9l2fg"]
Sep 15 20:05:18.831276 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.831259    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kszh"
Sep 15 20:05:18.834002 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.833982    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-hdqjv\""
Sep 15 20:05:18.834121 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.834021    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 15 20:05:18.834121 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.834072    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 15 20:05:18.841543 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.841048    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-2kszh"]
Sep 15 20:05:18.841543 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.841088    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-49mwc"]
Sep 15 20:05:18.841543 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.841191    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:05:18.844080 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.844058    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 15 20:05:18.844309 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.844285    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 15 20:05:18.844394 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.844368    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-g82mz\""
Sep 15 20:05:18.844761 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.844746    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 15 20:05:18.850421 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.850401    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 15 20:05:18.851855 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.851835    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-qg2bs"]
Sep 15 20:05:18.851968 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.851946    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-49mwc"
Sep 15 20:05:18.854776 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.854518    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 15 20:05:18.854776 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.854569    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 15 20:05:18.854776 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.854584    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-lzsdw\""
Sep 15 20:05:18.854776 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.854522    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 15 20:05:18.867685 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.867666    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-557757c5cd-9l2fg"]
Sep 15 20:05:18.867792 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.867700    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qg2bs"
Sep 15 20:05:18.867792 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.867708    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-49mwc"]
Sep 15 20:05:18.867792 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.867731    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qg2bs"]
Sep 15 20:05:18.870226 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.870154    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 15 20:05:18.870326 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.870187    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 15 20:05:18.870326 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.870244    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jwzj9\""
Sep 15 20:05:18.972201 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.972172    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-bound-sa-token\") pod \"image-registry-557757c5cd-9l2fg\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") " pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:05:18.972201 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.972202    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkjrv\" (UniqueName: \"kubernetes.io/projected/7a1d40a8-4253-43b6-b7c8-d278b7649dcd-kube-api-access-lkjrv\") pod \"ingress-canary-49mwc\" (UID: \"7a1d40a8-4253-43b6-b7c8-d278b7649dcd\") " pod="openshift-ingress-canary/ingress-canary-49mwc"
Sep 15 20:05:18.972475 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.972224    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0c65a532-9608-4066-9b1a-156727664921-metrics-tls\") pod \"dns-default-qg2bs\" (UID: \"0c65a532-9608-4066-9b1a-156727664921\") " pod="openshift-dns/dns-default-qg2bs"
Sep 15 20:05:18.972475 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.972274    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9hcc\" (UniqueName: \"kubernetes.io/projected/0c65a532-9608-4066-9b1a-156727664921-kube-api-access-r9hcc\") pod \"dns-default-qg2bs\" (UID: \"0c65a532-9608-4066-9b1a-156727664921\") " pod="openshift-dns/dns-default-qg2bs"
Sep 15 20:05:18.972475 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.972319    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/fcb6cd63-75f0-4efd-a878-fb7236f4109b-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-2kszh\" (UID: \"fcb6cd63-75f0-4efd-a878-fb7236f4109b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kszh"
Sep 15 20:05:18.972475 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.972345    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lw4gn\" (UniqueName: \"kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-kube-api-access-lw4gn\") pod \"image-registry-557757c5cd-9l2fg\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") " pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:05:18.972475 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.972390    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c65a532-9608-4066-9b1a-156727664921-tmp-dir\") pod \"dns-default-qg2bs\" (UID: \"0c65a532-9608-4066-9b1a-156727664921\") " pod="openshift-dns/dns-default-qg2bs"
Sep 15 20:05:18.972475 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.972411    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a1d40a8-4253-43b6-b7c8-d278b7649dcd-cert\") pod \"ingress-canary-49mwc\" (UID: \"7a1d40a8-4253-43b6-b7c8-d278b7649dcd\") " pod="openshift-ingress-canary/ingress-canary-49mwc"
Sep 15 20:05:18.972740 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.972479    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9cb71e8f-b673-49d6-bce3-2b205b80b22f-ca-trust-extracted\") pod \"image-registry-557757c5cd-9l2fg\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") " pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:05:18.972740 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.972508    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9cb71e8f-b673-49d6-bce3-2b205b80b22f-image-registry-private-configuration\") pod \"image-registry-557757c5cd-9l2fg\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") " pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:05:18.972740 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.972532    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-tls\") pod \"image-registry-557757c5cd-9l2fg\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") " pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:05:18.972740 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.972556    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9cb71e8f-b673-49d6-bce3-2b205b80b22f-installation-pull-secrets\") pod \"image-registry-557757c5cd-9l2fg\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") " pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:05:18.972740 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.972571    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0c65a532-9608-4066-9b1a-156727664921-config-volume\") pod \"dns-default-qg2bs\" (UID: \"0c65a532-9608-4066-9b1a-156727664921\") " pod="openshift-dns/dns-default-qg2bs"
Sep 15 20:05:18.972740 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.972603    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-certificates\") pod \"image-registry-557757c5cd-9l2fg\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") " pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:05:18.972740 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.972625    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9cb71e8f-b673-49d6-bce3-2b205b80b22f-trusted-ca\") pod \"image-registry-557757c5cd-9l2fg\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") " pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:05:18.972740 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.972656    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fcb6cd63-75f0-4efd-a878-fb7236f4109b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-2kszh\" (UID: \"fcb6cd63-75f0-4efd-a878-fb7236f4109b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kszh"
Sep 15 20:05:18.981884 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:18.981853    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:05:19.073144 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.073113    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9cb71e8f-b673-49d6-bce3-2b205b80b22f-image-registry-private-configuration\") pod \"image-registry-557757c5cd-9l2fg\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") " pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:05:19.073329 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.073152    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-tls\") pod \"image-registry-557757c5cd-9l2fg\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") " pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:05:19.073329 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.073178    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9cb71e8f-b673-49d6-bce3-2b205b80b22f-installation-pull-secrets\") pod \"image-registry-557757c5cd-9l2fg\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") " pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:05:19.073329 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.073202    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0c65a532-9608-4066-9b1a-156727664921-config-volume\") pod \"dns-default-qg2bs\" (UID: \"0c65a532-9608-4066-9b1a-156727664921\") " pod="openshift-dns/dns-default-qg2bs"
Sep 15 20:05:19.073329 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.073244    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-certificates\") pod \"image-registry-557757c5cd-9l2fg\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") " pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:05:19.073329 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.073269    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9cb71e8f-b673-49d6-bce3-2b205b80b22f-trusted-ca\") pod \"image-registry-557757c5cd-9l2fg\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") " pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:05:19.073329 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:19.073288    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 20:05:19.073329 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.073297    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fcb6cd63-75f0-4efd-a878-fb7236f4109b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-2kszh\" (UID: \"fcb6cd63-75f0-4efd-a878-fb7236f4109b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kszh"
Sep 15 20:05:19.073678 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.073352    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-bound-sa-token\") pod \"image-registry-557757c5cd-9l2fg\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") " pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:05:19.073678 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.073368    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkjrv\" (UniqueName: \"kubernetes.io/projected/7a1d40a8-4253-43b6-b7c8-d278b7649dcd-kube-api-access-lkjrv\") pod \"ingress-canary-49mwc\" (UID: \"7a1d40a8-4253-43b6-b7c8-d278b7649dcd\") " pod="openshift-ingress-canary/ingress-canary-49mwc"
Sep 15 20:05:19.073678 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.073400    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0c65a532-9608-4066-9b1a-156727664921-metrics-tls\") pod \"dns-default-qg2bs\" (UID: \"0c65a532-9608-4066-9b1a-156727664921\") " pod="openshift-dns/dns-default-qg2bs"
Sep 15 20:05:19.073678 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.073424    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9hcc\" (UniqueName: \"kubernetes.io/projected/0c65a532-9608-4066-9b1a-156727664921-kube-api-access-r9hcc\") pod \"dns-default-qg2bs\" (UID: \"0c65a532-9608-4066-9b1a-156727664921\") " pod="openshift-dns/dns-default-qg2bs"
Sep 15 20:05:19.073678 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.073453    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/fcb6cd63-75f0-4efd-a878-fb7236f4109b-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-2kszh\" (UID: \"fcb6cd63-75f0-4efd-a878-fb7236f4109b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kszh"
Sep 15 20:05:19.073678 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.073474    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lw4gn\" (UniqueName: \"kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-kube-api-access-lw4gn\") pod \"image-registry-557757c5cd-9l2fg\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") " pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:05:19.073678 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.073517    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c65a532-9608-4066-9b1a-156727664921-tmp-dir\") pod \"dns-default-qg2bs\" (UID: \"0c65a532-9608-4066-9b1a-156727664921\") " pod="openshift-dns/dns-default-qg2bs"
Sep 15 20:05:19.073678 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.073554    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a1d40a8-4253-43b6-b7c8-d278b7649dcd-cert\") pod \"ingress-canary-49mwc\" (UID: \"7a1d40a8-4253-43b6-b7c8-d278b7649dcd\") " pod="openshift-ingress-canary/ingress-canary-49mwc"
Sep 15 20:05:19.073678 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.073606    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9cb71e8f-b673-49d6-bce3-2b205b80b22f-ca-trust-extracted\") pod \"image-registry-557757c5cd-9l2fg\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") " pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:05:19.074037 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.073814    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0c65a532-9608-4066-9b1a-156727664921-config-volume\") pod \"dns-default-qg2bs\" (UID: \"0c65a532-9608-4066-9b1a-156727664921\") " pod="openshift-dns/dns-default-qg2bs"
Sep 15 20:05:19.074037 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.073906    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9cb71e8f-b673-49d6-bce3-2b205b80b22f-ca-trust-extracted\") pod \"image-registry-557757c5cd-9l2fg\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") " pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:05:19.074037 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.073944    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-certificates\") pod \"image-registry-557757c5cd-9l2fg\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") " pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:05:19.074037 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:19.073304    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-557757c5cd-9l2fg: secret "image-registry-tls" not found
Sep 15 20:05:19.074246 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:19.074053    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-tls podName:9cb71e8f-b673-49d6-bce3-2b205b80b22f nodeName:}" failed. No retries permitted until 2026-09-15 20:05:19.574025797 +0000 UTC m=+34.062490265 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-tls") pod "image-registry-557757c5cd-9l2fg" (UID: "9cb71e8f-b673-49d6-bce3-2b205b80b22f") : secret "image-registry-tls" not found
Sep 15 20:05:19.074326 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:19.074310    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 15 20:05:19.074376 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.074359    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c65a532-9608-4066-9b1a-156727664921-tmp-dir\") pod \"dns-default-qg2bs\" (UID: \"0c65a532-9608-4066-9b1a-156727664921\") " pod="openshift-dns/dns-default-qg2bs"
Sep 15 20:05:19.074421 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:19.073911    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 20:05:19.074466 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:19.074379    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcb6cd63-75f0-4efd-a878-fb7236f4109b-networking-console-plugin-cert podName:fcb6cd63-75f0-4efd-a878-fb7236f4109b nodeName:}" failed. No retries permitted until 2026-09-15 20:05:19.574359176 +0000 UTC m=+34.062823849 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/fcb6cd63-75f0-4efd-a878-fb7236f4109b-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-2kszh" (UID: "fcb6cd63-75f0-4efd-a878-fb7236f4109b") : secret "networking-console-plugin-cert" not found
Sep 15 20:05:19.074466 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:19.074455    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c65a532-9608-4066-9b1a-156727664921-metrics-tls podName:0c65a532-9608-4066-9b1a-156727664921 nodeName:}" failed. No retries permitted until 2026-09-15 20:05:19.574432589 +0000 UTC m=+34.062897058 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0c65a532-9608-4066-9b1a-156727664921-metrics-tls") pod "dns-default-qg2bs" (UID: "0c65a532-9608-4066-9b1a-156727664921") : secret "dns-default-metrics-tls" not found
Sep 15 20:05:19.074893 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.074864    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/fcb6cd63-75f0-4efd-a878-fb7236f4109b-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-2kszh\" (UID: \"fcb6cd63-75f0-4efd-a878-fb7236f4109b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kszh"
Sep 15 20:05:19.075368 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.075346    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9cb71e8f-b673-49d6-bce3-2b205b80b22f-trusted-ca\") pod \"image-registry-557757c5cd-9l2fg\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") " pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:05:19.078327 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.078305    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9cb71e8f-b673-49d6-bce3-2b205b80b22f-installation-pull-secrets\") pod \"image-registry-557757c5cd-9l2fg\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") " pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:05:19.078421 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.078326    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a1d40a8-4253-43b6-b7c8-d278b7649dcd-cert\") pod \"ingress-canary-49mwc\" (UID: \"7a1d40a8-4253-43b6-b7c8-d278b7649dcd\") " pod="openshift-ingress-canary/ingress-canary-49mwc"
Sep 15 20:05:19.078421 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.078329    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9cb71e8f-b673-49d6-bce3-2b205b80b22f-image-registry-private-configuration\") pod \"image-registry-557757c5cd-9l2fg\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") " pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:05:19.083708 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.083630    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:05:19.089032 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.086344    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-bound-sa-token\") pod \"image-registry-557757c5cd-9l2fg\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") " pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:05:19.089032 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.086398    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lw4gn\" (UniqueName: \"kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-kube-api-access-lw4gn\") pod \"image-registry-557757c5cd-9l2fg\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") " pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:05:19.089032 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.087195    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkjrv\" (UniqueName: \"kubernetes.io/projected/7a1d40a8-4253-43b6-b7c8-d278b7649dcd-kube-api-access-lkjrv\") pod \"ingress-canary-49mwc\" (UID: \"7a1d40a8-4253-43b6-b7c8-d278b7649dcd\") " pod="openshift-ingress-canary/ingress-canary-49mwc"
Sep 15 20:05:19.089782 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.089738    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9hcc\" (UniqueName: \"kubernetes.io/projected/0c65a532-9608-4066-9b1a-156727664921-kube-api-access-r9hcc\") pod \"dns-default-qg2bs\" (UID: \"0c65a532-9608-4066-9b1a-156727664921\") " pod="openshift-dns/dns-default-qg2bs"
Sep 15 20:05:19.137614 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.137585    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzw9m"
Sep 15 20:05:19.137771 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.137676    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:05:19.137849 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.137826    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7hmwc"
Sep 15 20:05:19.140451 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.140430    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 15 20:05:19.140693 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.140675    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 15 20:05:19.140693 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.140690    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 15 20:05:19.140825 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.140699    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fxg67\""
Sep 15 20:05:19.140825 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.140746    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 15 20:05:19.140825 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.140683    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l9zsw\""
Sep 15 20:05:19.163312 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.163295    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-49mwc"
Sep 15 20:05:19.452133 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.452106    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-49mwc"]
Sep 15 20:05:19.458099 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:05:19.456525    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a1d40a8_4253_43b6_b7c8_d278b7649dcd.slice/crio-8ca11c155be681dfc474b257cb3409af0f01eed79624a669337f6fbb1e6bd82a WatchSource:0}: Error finding container 8ca11c155be681dfc474b257cb3409af0f01eed79624a669337f6fbb1e6bd82a: Status 404 returned error can't find the container with id 8ca11c155be681dfc474b257cb3409af0f01eed79624a669337f6fbb1e6bd82a
Sep 15 20:05:19.578468 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.578386    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-tls\") pod \"image-registry-557757c5cd-9l2fg\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") " pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:05:19.578468 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.578459    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fcb6cd63-75f0-4efd-a878-fb7236f4109b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-2kszh\" (UID: \"fcb6cd63-75f0-4efd-a878-fb7236f4109b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kszh"
Sep 15 20:05:19.578700 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.578521    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0c65a532-9608-4066-9b1a-156727664921-metrics-tls\") pod \"dns-default-qg2bs\" (UID: \"0c65a532-9608-4066-9b1a-156727664921\") " pod="openshift-dns/dns-default-qg2bs"
Sep 15 20:05:19.578700 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:19.578554    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 20:05:19.578700 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:19.578575    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-557757c5cd-9l2fg: secret "image-registry-tls" not found
Sep 15 20:05:19.578700 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:19.578580    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 15 20:05:19.578700 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:19.578646    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 20:05:19.578700 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:19.578651    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-tls podName:9cb71e8f-b673-49d6-bce3-2b205b80b22f nodeName:}" failed. No retries permitted until 2026-09-15 20:05:20.578633978 +0000 UTC m=+35.067098436 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-tls") pod "image-registry-557757c5cd-9l2fg" (UID: "9cb71e8f-b673-49d6-bce3-2b205b80b22f") : secret "image-registry-tls" not found
Sep 15 20:05:19.579129 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:19.578712    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcb6cd63-75f0-4efd-a878-fb7236f4109b-networking-console-plugin-cert podName:fcb6cd63-75f0-4efd-a878-fb7236f4109b nodeName:}" failed. No retries permitted until 2026-09-15 20:05:20.578696041 +0000 UTC m=+35.067160502 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/fcb6cd63-75f0-4efd-a878-fb7236f4109b-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-2kszh" (UID: "fcb6cd63-75f0-4efd-a878-fb7236f4109b") : secret "networking-console-plugin-cert" not found
Sep 15 20:05:19.579129 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:19.578728    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c65a532-9608-4066-9b1a-156727664921-metrics-tls podName:0c65a532-9608-4066-9b1a-156727664921 nodeName:}" failed. No retries permitted until 2026-09-15 20:05:20.578718642 +0000 UTC m=+35.067183103 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0c65a532-9608-4066-9b1a-156727664921-metrics-tls") pod "dns-default-qg2bs" (UID: "0c65a532-9608-4066-9b1a-156727664921") : secret "dns-default-metrics-tls" not found
Sep 15 20:05:19.679630 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.679590    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-metrics-certs\") pod \"network-metrics-daemon-59mfs\" (UID: \"3b8d9342-6574-4d0d-a326-0299eb3b4c5d\") " pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:05:19.679780 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:19.679742    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 15 20:05:19.679840 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:19.679802    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-metrics-certs podName:3b8d9342-6574-4d0d-a326-0299eb3b4c5d nodeName:}" failed. No retries permitted until 2026-09-15 20:05:51.679785344 +0000 UTC m=+66.168249810 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-metrics-certs") pod "network-metrics-daemon-59mfs" (UID: "3b8d9342-6574-4d0d-a326-0299eb3b4c5d") : secret "metrics-daemon-secret" not found
Sep 15 20:05:19.780475 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.780441    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c3054ad-31af-4d06-9809-1eab5e2b140c-original-pull-secret\") pod \"global-pull-secret-syncer-7hmwc\" (UID: \"6c3054ad-31af-4d06-9809-1eab5e2b140c\") " pod="kube-system/global-pull-secret-syncer-7hmwc"
Sep 15 20:05:19.780628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.780503    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sblk8\" (UniqueName: \"kubernetes.io/projected/43a5c998-df74-4a93-b02f-3d53c8be805a-kube-api-access-sblk8\") pod \"network-check-target-gzw9m\" (UID: \"43a5c998-df74-4a93-b02f-3d53c8be805a\") " pod="openshift-network-diagnostics/network-check-target-gzw9m"
Sep 15 20:05:19.783950 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.783912    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sblk8\" (UniqueName: \"kubernetes.io/projected/43a5c998-df74-4a93-b02f-3d53c8be805a-kube-api-access-sblk8\") pod \"network-check-target-gzw9m\" (UID: \"43a5c998-df74-4a93-b02f-3d53c8be805a\") " pod="openshift-network-diagnostics/network-check-target-gzw9m"
Sep 15 20:05:19.784056 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:19.783911    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c3054ad-31af-4d06-9809-1eab5e2b140c-original-pull-secret\") pod \"global-pull-secret-syncer-7hmwc\" (UID: \"6c3054ad-31af-4d06-9809-1eab5e2b140c\") " pod="kube-system/global-pull-secret-syncer-7hmwc"
Sep 15 20:05:20.046868 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:20.046828    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzw9m"
Sep 15 20:05:20.057192 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:20.057165    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7hmwc"
Sep 15 20:05:20.288135 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:20.288092    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-49mwc" event={"ID":"7a1d40a8-4253-43b6-b7c8-d278b7649dcd","Type":"ContainerStarted","Data":"8ca11c155be681dfc474b257cb3409af0f01eed79624a669337f6fbb1e6bd82a"}
Sep 15 20:05:20.542114 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:20.542077    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:05:20.586880 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:20.586852    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fcb6cd63-75f0-4efd-a878-fb7236f4109b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-2kszh\" (UID: \"fcb6cd63-75f0-4efd-a878-fb7236f4109b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kszh"
Sep 15 20:05:20.587046 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:20.586907    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0c65a532-9608-4066-9b1a-156727664921-metrics-tls\") pod \"dns-default-qg2bs\" (UID: \"0c65a532-9608-4066-9b1a-156727664921\") " pod="openshift-dns/dns-default-qg2bs"
Sep 15 20:05:20.587046 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:20.587032    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 20:05:20.587159 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:20.587046    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-tls\") pod \"image-registry-557757c5cd-9l2fg\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") " pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:05:20.587159 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:20.587101    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c65a532-9608-4066-9b1a-156727664921-metrics-tls podName:0c65a532-9608-4066-9b1a-156727664921 nodeName:}" failed. No retries permitted until 2026-09-15 20:05:22.587080108 +0000 UTC m=+37.075544564 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0c65a532-9608-4066-9b1a-156727664921-metrics-tls") pod "dns-default-qg2bs" (UID: "0c65a532-9608-4066-9b1a-156727664921") : secret "dns-default-metrics-tls" not found
Sep 15 20:05:20.587159 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:20.587033    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 15 20:05:20.587159 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:20.587149    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 20:05:20.587339 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:20.587163    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-557757c5cd-9l2fg: secret "image-registry-tls" not found
Sep 15 20:05:20.587339 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:20.587164    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcb6cd63-75f0-4efd-a878-fb7236f4109b-networking-console-plugin-cert podName:fcb6cd63-75f0-4efd-a878-fb7236f4109b nodeName:}" failed. No retries permitted until 2026-09-15 20:05:22.58714695 +0000 UTC m=+37.075611411 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/fcb6cd63-75f0-4efd-a878-fb7236f4109b-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-2kszh" (UID: "fcb6cd63-75f0-4efd-a878-fb7236f4109b") : secret "networking-console-plugin-cert" not found
Sep 15 20:05:20.587339 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:20.587209    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-tls podName:9cb71e8f-b673-49d6-bce3-2b205b80b22f nodeName:}" failed. No retries permitted until 2026-09-15 20:05:22.587199781 +0000 UTC m=+37.075664235 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-tls") pod "image-registry-557757c5cd-9l2fg" (UID: "9cb71e8f-b673-49d6-bce3-2b205b80b22f") : secret "image-registry-tls" not found
Sep 15 20:05:20.655930 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:20.655856    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:05:21.484053 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:21.483989    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7hmwc"]
Sep 15 20:05:21.487913 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:21.487891    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gzw9m"]
Sep 15 20:05:21.561226 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:05:21.561197    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c3054ad_31af_4d06_9809_1eab5e2b140c.slice/crio-7bed225b8e965a734f9aec34fd34ca943db01d5eb26a7e67057855476343376e WatchSource:0}: Error finding container 7bed225b8e965a734f9aec34fd34ca943db01d5eb26a7e67057855476343376e: Status 404 returned error can't find the container with id 7bed225b8e965a734f9aec34fd34ca943db01d5eb26a7e67057855476343376e
Sep 15 20:05:21.561792 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:05:21.561771    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43a5c998_df74_4a93_b02f_3d53c8be805a.slice/crio-50ccdb293d26e0b0e411deee1153b07de9d7717d2d27dc34d4da5d272cc32d63 WatchSource:0}: Error finding container 50ccdb293d26e0b0e411deee1153b07de9d7717d2d27dc34d4da5d272cc32d63: Status 404 returned error can't find the container with id 50ccdb293d26e0b0e411deee1153b07de9d7717d2d27dc34d4da5d272cc32d63
Sep 15 20:05:22.299173 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:22.298903    2576 generic.go:356] "Generic (PLEG): container finished" podID="794a8849-e693-4fd1-905d-addff4740cda" containerID="35143db3100754b94ae6ed2df06fe9d6c03c50ad54bab7f603258793aa0bcbf4" exitCode=0
Sep 15 20:05:22.299343 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:22.299099    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9r4qs" event={"ID":"794a8849-e693-4fd1-905d-addff4740cda","Type":"ContainerDied","Data":"35143db3100754b94ae6ed2df06fe9d6c03c50ad54bab7f603258793aa0bcbf4"}
Sep 15 20:05:22.308148 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:22.308118    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gzw9m" event={"ID":"43a5c998-df74-4a93-b02f-3d53c8be805a","Type":"ContainerStarted","Data":"50ccdb293d26e0b0e411deee1153b07de9d7717d2d27dc34d4da5d272cc32d63"}
Sep 15 20:05:22.314950 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:22.314268    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7hmwc" event={"ID":"6c3054ad-31af-4d06-9809-1eab5e2b140c","Type":"ContainerStarted","Data":"7bed225b8e965a734f9aec34fd34ca943db01d5eb26a7e67057855476343376e"}
Sep 15 20:05:22.602025 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:22.601989    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fcb6cd63-75f0-4efd-a878-fb7236f4109b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-2kszh\" (UID: \"fcb6cd63-75f0-4efd-a878-fb7236f4109b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kszh"
Sep 15 20:05:22.602680 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:22.602062    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0c65a532-9608-4066-9b1a-156727664921-metrics-tls\") pod \"dns-default-qg2bs\" (UID: \"0c65a532-9608-4066-9b1a-156727664921\") " pod="openshift-dns/dns-default-qg2bs"
Sep 15 20:05:22.602680 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:22.602118    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-tls\") pod \"image-registry-557757c5cd-9l2fg\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") " pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:05:22.602680 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:22.602220    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 20:05:22.602680 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:22.602227    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 20:05:22.602680 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:22.602240    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 15 20:05:22.602680 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:22.602234    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-557757c5cd-9l2fg: secret "image-registry-tls" not found
Sep 15 20:05:22.602680 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:22.602300    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcb6cd63-75f0-4efd-a878-fb7236f4109b-networking-console-plugin-cert podName:fcb6cd63-75f0-4efd-a878-fb7236f4109b nodeName:}" failed. No retries permitted until 2026-09-15 20:05:26.602279509 +0000 UTC m=+41.090743984 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/fcb6cd63-75f0-4efd-a878-fb7236f4109b-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-2kszh" (UID: "fcb6cd63-75f0-4efd-a878-fb7236f4109b") : secret "networking-console-plugin-cert" not found
Sep 15 20:05:22.602680 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:22.602317    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c65a532-9608-4066-9b1a-156727664921-metrics-tls podName:0c65a532-9608-4066-9b1a-156727664921 nodeName:}" failed. No retries permitted until 2026-09-15 20:05:26.602307825 +0000 UTC m=+41.090772282 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0c65a532-9608-4066-9b1a-156727664921-metrics-tls") pod "dns-default-qg2bs" (UID: "0c65a532-9608-4066-9b1a-156727664921") : secret "dns-default-metrics-tls" not found
Sep 15 20:05:22.602680 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:22.602329    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-tls podName:9cb71e8f-b673-49d6-bce3-2b205b80b22f nodeName:}" failed. No retries permitted until 2026-09-15 20:05:26.602322474 +0000 UTC m=+41.090786934 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-tls") pod "image-registry-557757c5cd-9l2fg" (UID: "9cb71e8f-b673-49d6-bce3-2b205b80b22f") : secret "image-registry-tls" not found
Sep 15 20:05:23.323233 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:23.322332    2576 generic.go:356] "Generic (PLEG): container finished" podID="794a8849-e693-4fd1-905d-addff4740cda" containerID="f1bdaf99c7b6f3ddf45759b5aa264b308e21ecf290676f9355a0e1d76e1c4f31" exitCode=0
Sep 15 20:05:23.323233 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:23.322405    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9r4qs" event={"ID":"794a8849-e693-4fd1-905d-addff4740cda","Type":"ContainerDied","Data":"f1bdaf99c7b6f3ddf45759b5aa264b308e21ecf290676f9355a0e1d76e1c4f31"}
Sep 15 20:05:24.435944 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:24.435840    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:05:24.508368 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:24.508333    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:05:26.634659 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:26.634624    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-tls\") pod \"image-registry-557757c5cd-9l2fg\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") " pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:05:26.635079 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:26.634676    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fcb6cd63-75f0-4efd-a878-fb7236f4109b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-2kszh\" (UID: \"fcb6cd63-75f0-4efd-a878-fb7236f4109b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kszh"
Sep 15 20:05:26.635079 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:26.634735    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0c65a532-9608-4066-9b1a-156727664921-metrics-tls\") pod \"dns-default-qg2bs\" (UID: \"0c65a532-9608-4066-9b1a-156727664921\") " pod="openshift-dns/dns-default-qg2bs"
Sep 15 20:05:26.635079 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:26.634797    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 20:05:26.635079 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:26.634821    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-557757c5cd-9l2fg: secret "image-registry-tls" not found
Sep 15 20:05:26.635079 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:26.634853    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 15 20:05:26.635079 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:26.634858    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 20:05:26.635079 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:26.634903    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-tls podName:9cb71e8f-b673-49d6-bce3-2b205b80b22f nodeName:}" failed. No retries permitted until 2026-09-15 20:05:34.634880408 +0000 UTC m=+49.123344880 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-tls") pod "image-registry-557757c5cd-9l2fg" (UID: "9cb71e8f-b673-49d6-bce3-2b205b80b22f") : secret "image-registry-tls" not found
Sep 15 20:05:26.635079 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:26.634939    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcb6cd63-75f0-4efd-a878-fb7236f4109b-networking-console-plugin-cert podName:fcb6cd63-75f0-4efd-a878-fb7236f4109b nodeName:}" failed. No retries permitted until 2026-09-15 20:05:34.634911468 +0000 UTC m=+49.123375925 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/fcb6cd63-75f0-4efd-a878-fb7236f4109b-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-2kszh" (UID: "fcb6cd63-75f0-4efd-a878-fb7236f4109b") : secret "networking-console-plugin-cert" not found
Sep 15 20:05:26.635079 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:26.634954    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c65a532-9608-4066-9b1a-156727664921-metrics-tls podName:0c65a532-9608-4066-9b1a-156727664921 nodeName:}" failed. No retries permitted until 2026-09-15 20:05:34.634946762 +0000 UTC m=+49.123411216 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0c65a532-9608-4066-9b1a-156727664921-metrics-tls") pod "dns-default-qg2bs" (UID: "0c65a532-9608-4066-9b1a-156727664921") : secret "dns-default-metrics-tls" not found
Sep 15 20:05:27.278663 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:27.278638    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:05:27.376066 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:27.376020    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:05:27.471474 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:27.471431    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:05:28.335358 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:28.335324    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9r4qs" event={"ID":"794a8849-e693-4fd1-905d-addff4740cda","Type":"ContainerStarted","Data":"4aeea219126fb73d1a4646f82e2009c26fa749954803d92e69949ab568bd382c"}
Sep 15 20:05:28.336799 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:28.336780    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gzw9m" event={"ID":"43a5c998-df74-4a93-b02f-3d53c8be805a","Type":"ContainerStarted","Data":"2ec5cf7601d489a36dad6d60929c33a955c1c77e491461067e514d16ea70735b"}
Sep 15 20:05:28.336879 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:28.336850    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-gzw9m"
Sep 15 20:05:28.338213 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:28.338187    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-49mwc" event={"ID":"7a1d40a8-4253-43b6-b7c8-d278b7649dcd","Type":"ContainerStarted","Data":"a3739ce7754709cfb152b0c18ba47cd0f84252c50c301ab518c44a4f922c0e87"}
Sep 15 20:05:28.339479 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:28.339453    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7hmwc" event={"ID":"6c3054ad-31af-4d06-9809-1eab5e2b140c","Type":"ContainerStarted","Data":"6cb194969ad66f0476b361f481be9bd231ada6448a616a389b6e9014cfb77732"}
Sep 15 20:05:28.345382 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:28.345357    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:05:28.358078 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:28.358040    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9r4qs" podStartSLOduration=11.493742754 podStartE2EDuration="42.358028568s" totalImagesPullingTime="30.864285814s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:04:46 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:05:28.357942842 +0000 UTC m=+42.846407321" watchObservedRunningTime="2026-09-15 20:05:28.358028568 +0000 UTC m=+42.846493043"
Sep 15 20:05:28.374714 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:28.374682    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-gzw9m" podStartSLOduration=36.723273092 podStartE2EDuration="42.374670425s" totalImagesPullingTime="5.651397333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:05:28.374198516 +0000 UTC m=+42.862662995" watchObservedRunningTime="2026-09-15 20:05:28.374670425 +0000 UTC m=+42.863134927"
Sep 15 20:05:28.391778 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:28.391740    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-49mwc" podStartSLOduration=2.620426821 podStartE2EDuration="10.391725917s" totalImagesPullingTime="7.771299096s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:05:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:05:28.391650823 +0000 UTC m=+42.880115299" watchObservedRunningTime="2026-09-15 20:05:28.391725917 +0000 UTC m=+42.880190397"
Sep 15 20:05:28.410067 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:28.410034    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7hmwc" podStartSLOduration=36.74865908 podStartE2EDuration="42.410022647s" totalImagesPullingTime="5.661363567s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:05:28.409141846 +0000 UTC m=+42.897606322" watchObservedRunningTime="2026-09-15 20:05:28.410022647 +0000 UTC m=+42.898487122"
Sep 15 20:05:28.416057 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:28.416029    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:05:28.783174 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:28.783141    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:05:28.850278 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:28.850241    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:05:29.139787 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:29.139697    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:05:32.581224 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:32.581189    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:05:32.682337 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:32.682309    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:05:33.029293 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:33.029265    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:05:34.691211 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:34.691174    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fcb6cd63-75f0-4efd-a878-fb7236f4109b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-2kszh\" (UID: \"fcb6cd63-75f0-4efd-a878-fb7236f4109b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kszh"
Sep 15 20:05:34.691652 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:34.691224    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0c65a532-9608-4066-9b1a-156727664921-metrics-tls\") pod \"dns-default-qg2bs\" (UID: \"0c65a532-9608-4066-9b1a-156727664921\") " pod="openshift-dns/dns-default-qg2bs"
Sep 15 20:05:34.691652 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:34.691261    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-tls\") pod \"image-registry-557757c5cd-9l2fg\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") " pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:05:34.691652 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:34.691327    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 15 20:05:34.691652 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:34.691349    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 20:05:34.691652 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:34.691359    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-557757c5cd-9l2fg: secret "image-registry-tls" not found
Sep 15 20:05:34.691652 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:34.691394    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcb6cd63-75f0-4efd-a878-fb7236f4109b-networking-console-plugin-cert podName:fcb6cd63-75f0-4efd-a878-fb7236f4109b nodeName:}" failed. No retries permitted until 2026-09-15 20:05:50.691376714 +0000 UTC m=+65.179841171 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/fcb6cd63-75f0-4efd-a878-fb7236f4109b-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-2kszh" (UID: "fcb6cd63-75f0-4efd-a878-fb7236f4109b") : secret "networking-console-plugin-cert" not found
Sep 15 20:05:34.691652 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:34.691403    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 20:05:34.691652 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:34.691409    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-tls podName:9cb71e8f-b673-49d6-bce3-2b205b80b22f nodeName:}" failed. No retries permitted until 2026-09-15 20:05:50.691402586 +0000 UTC m=+65.179867040 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-tls") pod "image-registry-557757c5cd-9l2fg" (UID: "9cb71e8f-b673-49d6-bce3-2b205b80b22f") : secret "image-registry-tls" not found
Sep 15 20:05:34.691652 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:34.691461    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c65a532-9608-4066-9b1a-156727664921-metrics-tls podName:0c65a532-9608-4066-9b1a-156727664921 nodeName:}" failed. No retries permitted until 2026-09-15 20:05:50.691444199 +0000 UTC m=+65.179908655 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0c65a532-9608-4066-9b1a-156727664921-metrics-tls") pod "dns-default-qg2bs" (UID: "0c65a532-9608-4066-9b1a-156727664921") : secret "dns-default-metrics-tls" not found
Sep 15 20:05:40.561545 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:40.561501    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:05:40.787523 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:40.787481    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:05:44.286088 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:44.286062    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6st24"
Sep 15 20:05:44.434118 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:44.434079    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:05:44.505137 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:44.505095    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:05:44.774901 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:44.774868    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:05:44.876250 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:44.876225    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:05:45.374363 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:45.374333    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:05:45.475474 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:45.475443    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:05:50.696906 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:50.696871    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-tls\") pod \"image-registry-557757c5cd-9l2fg\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") " pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:05:50.696906 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:50.696913    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fcb6cd63-75f0-4efd-a878-fb7236f4109b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-2kszh\" (UID: \"fcb6cd63-75f0-4efd-a878-fb7236f4109b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kszh"
Sep 15 20:05:50.697345 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:50.697013    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 15 20:05:50.697345 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:50.697068    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcb6cd63-75f0-4efd-a878-fb7236f4109b-networking-console-plugin-cert podName:fcb6cd63-75f0-4efd-a878-fb7236f4109b nodeName:}" failed. No retries permitted until 2026-09-15 20:06:22.697053807 +0000 UTC m=+97.185518262 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/fcb6cd63-75f0-4efd-a878-fb7236f4109b-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-2kszh" (UID: "fcb6cd63-75f0-4efd-a878-fb7236f4109b") : secret "networking-console-plugin-cert" not found
Sep 15 20:05:50.697345 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:50.697075    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 20:05:50.697345 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:50.697092    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-557757c5cd-9l2fg: secret "image-registry-tls" not found
Sep 15 20:05:50.697345 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:50.697145    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-tls podName:9cb71e8f-b673-49d6-bce3-2b205b80b22f nodeName:}" failed. No retries permitted until 2026-09-15 20:06:22.697131897 +0000 UTC m=+97.185596351 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-tls") pod "image-registry-557757c5cd-9l2fg" (UID: "9cb71e8f-b673-49d6-bce3-2b205b80b22f") : secret "image-registry-tls" not found
Sep 15 20:05:50.697345 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:50.697184    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0c65a532-9608-4066-9b1a-156727664921-metrics-tls\") pod \"dns-default-qg2bs\" (UID: \"0c65a532-9608-4066-9b1a-156727664921\") " pod="openshift-dns/dns-default-qg2bs"
Sep 15 20:05:50.697345 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:50.697256    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 20:05:50.697345 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:50.697287    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c65a532-9608-4066-9b1a-156727664921-metrics-tls podName:0c65a532-9608-4066-9b1a-156727664921 nodeName:}" failed. No retries permitted until 2026-09-15 20:06:22.697280001 +0000 UTC m=+97.185744455 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0c65a532-9608-4066-9b1a-156727664921-metrics-tls") pod "dns-default-qg2bs" (UID: "0c65a532-9608-4066-9b1a-156727664921") : secret "dns-default-metrics-tls" not found
Sep 15 20:05:51.702614 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:51.702553    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-metrics-certs\") pod \"network-metrics-daemon-59mfs\" (UID: \"3b8d9342-6574-4d0d-a326-0299eb3b4c5d\") " pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:05:51.703029 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:51.702718    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 15 20:05:51.703029 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:05:51.702790    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-metrics-certs podName:3b8d9342-6574-4d0d-a326-0299eb3b4c5d nodeName:}" failed. No retries permitted until 2026-09-15 20:06:55.702774641 +0000 UTC m=+130.191239095 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-metrics-certs") pod "network-metrics-daemon-59mfs" (UID: "3b8d9342-6574-4d0d-a326-0299eb3b4c5d") : secret "metrics-daemon-secret" not found
Sep 15 20:05:53.404444 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:53.404411    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:05:53.505238 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:53.505204    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:05:59.344553 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:05:59.344511    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-gzw9m"
Sep 15 20:06:06.344812 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:06:06.344778    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:06:06.417596 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:06:06.417563    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:06:22.700155 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:06:22.700122    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-tls\") pod \"image-registry-557757c5cd-9l2fg\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") " pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:06:22.700563 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:06:22.700166    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fcb6cd63-75f0-4efd-a878-fb7236f4109b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-2kszh\" (UID: \"fcb6cd63-75f0-4efd-a878-fb7236f4109b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kszh"
Sep 15 20:06:22.700563 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:06:22.700257    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 15 20:06:22.700563 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:06:22.700262    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 20:06:22.700563 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:06:22.700284    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-557757c5cd-9l2fg: secret "image-registry-tls" not found
Sep 15 20:06:22.700563 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:06:22.700307    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcb6cd63-75f0-4efd-a878-fb7236f4109b-networking-console-plugin-cert podName:fcb6cd63-75f0-4efd-a878-fb7236f4109b nodeName:}" failed. No retries permitted until 2026-09-15 20:07:26.700293039 +0000 UTC m=+161.188757506 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/fcb6cd63-75f0-4efd-a878-fb7236f4109b-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-2kszh" (UID: "fcb6cd63-75f0-4efd-a878-fb7236f4109b") : secret "networking-console-plugin-cert" not found
Sep 15 20:06:22.700563 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:06:22.700304    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0c65a532-9608-4066-9b1a-156727664921-metrics-tls\") pod \"dns-default-qg2bs\" (UID: \"0c65a532-9608-4066-9b1a-156727664921\") " pod="openshift-dns/dns-default-qg2bs"
Sep 15 20:06:22.700563 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:06:22.700328    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-tls podName:9cb71e8f-b673-49d6-bce3-2b205b80b22f nodeName:}" failed. No retries permitted until 2026-09-15 20:07:26.700316577 +0000 UTC m=+161.188781031 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-tls") pod "image-registry-557757c5cd-9l2fg" (UID: "9cb71e8f-b673-49d6-bce3-2b205b80b22f") : secret "image-registry-tls" not found
Sep 15 20:06:22.700563 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:06:22.700402    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 20:06:22.700563 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:06:22.700462    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c65a532-9608-4066-9b1a-156727664921-metrics-tls podName:0c65a532-9608-4066-9b1a-156727664921 nodeName:}" failed. No retries permitted until 2026-09-15 20:07:26.70044766 +0000 UTC m=+161.188912114 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0c65a532-9608-4066-9b1a-156727664921-metrics-tls") pod "dns-default-qg2bs" (UID: "0c65a532-9608-4066-9b1a-156727664921") : secret "dns-default-metrics-tls" not found
Sep 15 20:06:42.903491 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:06:42.903458    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:06:55.719838 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:06:55.719784    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-metrics-certs\") pod \"network-metrics-daemon-59mfs\" (UID: \"3b8d9342-6574-4d0d-a326-0299eb3b4c5d\") " pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:06:55.720338 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:06:55.719963    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 15 20:06:55.720338 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:06:55.720038    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-metrics-certs podName:3b8d9342-6574-4d0d-a326-0299eb3b4c5d nodeName:}" failed. No retries permitted until 2026-09-15 20:08:57.720021514 +0000 UTC m=+252.208485973 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-metrics-certs") pod "network-metrics-daemon-59mfs" (UID: "3b8d9342-6574-4d0d-a326-0299eb3b4c5d") : secret "metrics-daemon-secret" not found
Sep 15 20:07:21.844865 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:07:21.844827    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kszh" podUID="fcb6cd63-75f0-4efd-a878-fb7236f4109b"
Sep 15 20:07:21.855171 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:07:21.855149    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-557757c5cd-9l2fg" podUID="9cb71e8f-b673-49d6-bce3-2b205b80b22f"
Sep 15 20:07:21.877325 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:07:21.877300    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-qg2bs" podUID="0c65a532-9608-4066-9b1a-156727664921"
Sep 15 20:07:22.152705 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:07:22.152630    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-59mfs" podUID="3b8d9342-6574-4d0d-a326-0299eb3b4c5d"
Sep 15 20:07:22.599125 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:07:22.599097    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kszh"
Sep 15 20:07:22.599243 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:07:22.599127    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:07:22.599243 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:07:22.599241    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qg2bs"
Sep 15 20:07:26.720698 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:07:26.720659    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-tls\") pod \"image-registry-557757c5cd-9l2fg\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") " pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:07:26.721111 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:07:26.720703    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fcb6cd63-75f0-4efd-a878-fb7236f4109b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-2kszh\" (UID: \"fcb6cd63-75f0-4efd-a878-fb7236f4109b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kszh"
Sep 15 20:07:26.721111 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:07:26.720735    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0c65a532-9608-4066-9b1a-156727664921-metrics-tls\") pod \"dns-default-qg2bs\" (UID: \"0c65a532-9608-4066-9b1a-156727664921\") " pod="openshift-dns/dns-default-qg2bs"
Sep 15 20:07:26.721111 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:07:26.720810    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 20:07:26.721111 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:07:26.720860    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c65a532-9608-4066-9b1a-156727664921-metrics-tls podName:0c65a532-9608-4066-9b1a-156727664921 nodeName:}" failed. No retries permitted until 2026-09-15 20:09:28.72084648 +0000 UTC m=+283.209310948 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0c65a532-9608-4066-9b1a-156727664921-metrics-tls") pod "dns-default-qg2bs" (UID: "0c65a532-9608-4066-9b1a-156727664921") : secret "dns-default-metrics-tls" not found
Sep 15 20:07:26.721111 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:07:26.720808    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 20:07:26.721111 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:07:26.720899    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-557757c5cd-9l2fg: secret "image-registry-tls" not found
Sep 15 20:07:26.721111 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:07:26.720810    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 15 20:07:26.721111 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:07:26.720968    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-tls podName:9cb71e8f-b673-49d6-bce3-2b205b80b22f nodeName:}" failed. No retries permitted until 2026-09-15 20:09:28.720954054 +0000 UTC m=+283.209418521 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-tls") pod "image-registry-557757c5cd-9l2fg" (UID: "9cb71e8f-b673-49d6-bce3-2b205b80b22f") : secret "image-registry-tls" not found
Sep 15 20:07:26.721111 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:07:26.721008    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcb6cd63-75f0-4efd-a878-fb7236f4109b-networking-console-plugin-cert podName:fcb6cd63-75f0-4efd-a878-fb7236f4109b nodeName:}" failed. No retries permitted until 2026-09-15 20:09:28.720987609 +0000 UTC m=+283.209452073 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/fcb6cd63-75f0-4efd-a878-fb7236f4109b-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-2kszh" (UID: "fcb6cd63-75f0-4efd-a878-fb7236f4109b") : secret "networking-console-plugin-cert" not found
Sep 15 20:07:34.137372 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:07:34.137333    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:08:52.096296 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.096263    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-9cxxk"]
Sep 15 20:08:52.098060 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.098045    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-9cxxk"
Sep 15 20:08:52.100724 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.100703    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 15 20:08:52.100826 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.100704    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 15 20:08:52.101130 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.101116    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 15 20:08:52.101764 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.101738    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 15 20:08:52.102032 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.102016    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-c2wcs\""
Sep 15 20:08:52.105755 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.105739    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 15 20:08:52.113362 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.113343    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-9cxxk"]
Sep 15 20:08:52.114372 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.114355    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-wdnhk"]
Sep 15 20:08:52.115932 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.115903    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s6c6z"]
Sep 15 20:08:52.116046 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.116027    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-wdnhk"
Sep 15 20:08:52.117543 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.117525    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-jcrqk"]
Sep 15 20:08:52.117642 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.117626    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s6c6z"
Sep 15 20:08:52.118838 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.118822    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jcrqk"
Sep 15 20:08:52.120061 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.120045    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 15 20:08:52.120136 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.120085    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 15 20:08:52.120337 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.120318    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 15 20:08:52.120854 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.120374    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 15 20:08:52.120960 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.120861    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 15 20:08:52.125372 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.123570    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 15 20:08:52.125372 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.124141    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 15 20:08:52.125372 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.124364    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 15 20:08:52.125372 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.124690    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-l5glk\""
Sep 15 20:08:52.125372 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.125025    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 15 20:08:52.125372 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.125109    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 15 20:08:52.125890 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.125869    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-wdtfv\""
Sep 15 20:08:52.125981 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.125965    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-p6nxl\""
Sep 15 20:08:52.126058 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.126038    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 15 20:08:52.128783 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.128765    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s6c6z"]
Sep 15 20:08:52.132584 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.132565    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-jcrqk"]
Sep 15 20:08:52.140991 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.140972    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-wdnhk"]
Sep 15 20:08:52.178295 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.178269    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8r8c\" (UniqueName: \"kubernetes.io/projected/aea61046-fabb-4705-be56-2894d6e3ce75-kube-api-access-g8r8c\") pod \"cluster-samples-operator-86499c89c4-s6c6z\" (UID: \"aea61046-fabb-4705-be56-2894d6e3ce75\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s6c6z"
Sep 15 20:08:52.178385 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.178310    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eaa018c5-5f77-486a-a95d-aa730c416c06-config\") pod \"service-ca-operator-cf7c776b7-jcrqk\" (UID: \"eaa018c5-5f77-486a-a95d-aa730c416c06\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jcrqk"
Sep 15 20:08:52.178385 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.178334    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93c1a30d-050c-491d-adbc-b5e876968efd-trusted-ca-bundle\") pod \"insights-operator-5844b856db-9cxxk\" (UID: \"93c1a30d-050c-491d-adbc-b5e876968efd\") " pod="openshift-insights/insights-operator-5844b856db-9cxxk"
Sep 15 20:08:52.178453 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.178391    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gb7rt\" (UniqueName: \"kubernetes.io/projected/c085a1d9-7bd7-4769-9b58-c554276913f8-kube-api-access-gb7rt\") pod \"cluster-monitoring-operator-6cd954b897-wdnhk\" (UID: \"c085a1d9-7bd7-4769-9b58-c554276913f8\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-wdnhk"
Sep 15 20:08:52.178487 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.178466    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c085a1d9-7bd7-4769-9b58-c554276913f8-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-wdnhk\" (UID: \"c085a1d9-7bd7-4769-9b58-c554276913f8\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-wdnhk"
Sep 15 20:08:52.178487 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.178484    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eaa018c5-5f77-486a-a95d-aa730c416c06-serving-cert\") pod \"service-ca-operator-cf7c776b7-jcrqk\" (UID: \"eaa018c5-5f77-486a-a95d-aa730c416c06\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jcrqk"
Sep 15 20:08:52.178560 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.178500    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c085a1d9-7bd7-4769-9b58-c554276913f8-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-wdnhk\" (UID: \"c085a1d9-7bd7-4769-9b58-c554276913f8\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-wdnhk"
Sep 15 20:08:52.178560 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.178518    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/aea61046-fabb-4705-be56-2894d6e3ce75-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-s6c6z\" (UID: \"aea61046-fabb-4705-be56-2894d6e3ce75\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s6c6z"
Sep 15 20:08:52.178621 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.178550    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8gdf\" (UniqueName: \"kubernetes.io/projected/93c1a30d-050c-491d-adbc-b5e876968efd-kube-api-access-x8gdf\") pod \"insights-operator-5844b856db-9cxxk\" (UID: \"93c1a30d-050c-491d-adbc-b5e876968efd\") " pod="openshift-insights/insights-operator-5844b856db-9cxxk"
Sep 15 20:08:52.178621 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.178585    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rt9kq\" (UniqueName: \"kubernetes.io/projected/eaa018c5-5f77-486a-a95d-aa730c416c06-kube-api-access-rt9kq\") pod \"service-ca-operator-cf7c776b7-jcrqk\" (UID: \"eaa018c5-5f77-486a-a95d-aa730c416c06\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jcrqk"
Sep 15 20:08:52.178621 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.178603    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/93c1a30d-050c-491d-adbc-b5e876968efd-tmp\") pod \"insights-operator-5844b856db-9cxxk\" (UID: \"93c1a30d-050c-491d-adbc-b5e876968efd\") " pod="openshift-insights/insights-operator-5844b856db-9cxxk"
Sep 15 20:08:52.178710 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.178637    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93c1a30d-050c-491d-adbc-b5e876968efd-service-ca-bundle\") pod \"insights-operator-5844b856db-9cxxk\" (UID: \"93c1a30d-050c-491d-adbc-b5e876968efd\") " pod="openshift-insights/insights-operator-5844b856db-9cxxk"
Sep 15 20:08:52.178710 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.178652    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/93c1a30d-050c-491d-adbc-b5e876968efd-serving-cert\") pod \"insights-operator-5844b856db-9cxxk\" (UID: \"93c1a30d-050c-491d-adbc-b5e876968efd\") " pod="openshift-insights/insights-operator-5844b856db-9cxxk"
Sep 15 20:08:52.178710 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.178667    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/93c1a30d-050c-491d-adbc-b5e876968efd-snapshots\") pod \"insights-operator-5844b856db-9cxxk\" (UID: \"93c1a30d-050c-491d-adbc-b5e876968efd\") " pod="openshift-insights/insights-operator-5844b856db-9cxxk"
Sep 15 20:08:52.204615 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.204593    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-b6b4698b-54vk2"]
Sep 15 20:08:52.206510 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.206497    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-b6b4698b-54vk2"
Sep 15 20:08:52.211908 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.211892    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 15 20:08:52.212062 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.212039    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 15 20:08:52.212146 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.212066    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 15 20:08:52.212146 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.212078    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 15 20:08:52.212358 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.212168    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 15 20:08:52.212358 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.212231    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-fgx2z\""
Sep 15 20:08:52.216723 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.216708    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 15 20:08:52.234819 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.234792    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-b6b4698b-54vk2"]
Sep 15 20:08:52.279150 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.279128    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eaa018c5-5f77-486a-a95d-aa730c416c06-config\") pod \"service-ca-operator-cf7c776b7-jcrqk\" (UID: \"eaa018c5-5f77-486a-a95d-aa730c416c06\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jcrqk"
Sep 15 20:08:52.279282 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.279158    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/085814ed-8800-43b7-bc59-b4f46cf14608-service-ca-bundle\") pod \"router-default-b6b4698b-54vk2\" (UID: \"085814ed-8800-43b7-bc59-b4f46cf14608\") " pod="openshift-ingress/router-default-b6b4698b-54vk2"
Sep 15 20:08:52.279282 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.279189    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93c1a30d-050c-491d-adbc-b5e876968efd-trusted-ca-bundle\") pod \"insights-operator-5844b856db-9cxxk\" (UID: \"93c1a30d-050c-491d-adbc-b5e876968efd\") " pod="openshift-insights/insights-operator-5844b856db-9cxxk"
Sep 15 20:08:52.279282 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.279216    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/085814ed-8800-43b7-bc59-b4f46cf14608-default-certificate\") pod \"router-default-b6b4698b-54vk2\" (UID: \"085814ed-8800-43b7-bc59-b4f46cf14608\") " pod="openshift-ingress/router-default-b6b4698b-54vk2"
Sep 15 20:08:52.279282 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.279241    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb7rt\" (UniqueName: \"kubernetes.io/projected/c085a1d9-7bd7-4769-9b58-c554276913f8-kube-api-access-gb7rt\") pod \"cluster-monitoring-operator-6cd954b897-wdnhk\" (UID: \"c085a1d9-7bd7-4769-9b58-c554276913f8\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-wdnhk"
Sep 15 20:08:52.279282 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.279264    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/085814ed-8800-43b7-bc59-b4f46cf14608-stats-auth\") pod \"router-default-b6b4698b-54vk2\" (UID: \"085814ed-8800-43b7-bc59-b4f46cf14608\") " pod="openshift-ingress/router-default-b6b4698b-54vk2"
Sep 15 20:08:52.279587 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.279332    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/085814ed-8800-43b7-bc59-b4f46cf14608-metrics-certs\") pod \"router-default-b6b4698b-54vk2\" (UID: \"085814ed-8800-43b7-bc59-b4f46cf14608\") " pod="openshift-ingress/router-default-b6b4698b-54vk2"
Sep 15 20:08:52.279587 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.279368    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xvnb\" (UniqueName: \"kubernetes.io/projected/085814ed-8800-43b7-bc59-b4f46cf14608-kube-api-access-4xvnb\") pod \"router-default-b6b4698b-54vk2\" (UID: \"085814ed-8800-43b7-bc59-b4f46cf14608\") " pod="openshift-ingress/router-default-b6b4698b-54vk2"
Sep 15 20:08:52.279587 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.279407    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c085a1d9-7bd7-4769-9b58-c554276913f8-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-wdnhk\" (UID: \"c085a1d9-7bd7-4769-9b58-c554276913f8\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-wdnhk"
Sep 15 20:08:52.279587 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.279428    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eaa018c5-5f77-486a-a95d-aa730c416c06-serving-cert\") pod \"service-ca-operator-cf7c776b7-jcrqk\" (UID: \"eaa018c5-5f77-486a-a95d-aa730c416c06\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jcrqk"
Sep 15 20:08:52.279587 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.279458    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c085a1d9-7bd7-4769-9b58-c554276913f8-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-wdnhk\" (UID: \"c085a1d9-7bd7-4769-9b58-c554276913f8\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-wdnhk"
Sep 15 20:08:52.279587 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.279510    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/aea61046-fabb-4705-be56-2894d6e3ce75-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-s6c6z\" (UID: \"aea61046-fabb-4705-be56-2894d6e3ce75\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s6c6z"
Sep 15 20:08:52.279587 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.279538    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8gdf\" (UniqueName: \"kubernetes.io/projected/93c1a30d-050c-491d-adbc-b5e876968efd-kube-api-access-x8gdf\") pod \"insights-operator-5844b856db-9cxxk\" (UID: \"93c1a30d-050c-491d-adbc-b5e876968efd\") " pod="openshift-insights/insights-operator-5844b856db-9cxxk"
Sep 15 20:08:52.279587 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.279565    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rt9kq\" (UniqueName: \"kubernetes.io/projected/eaa018c5-5f77-486a-a95d-aa730c416c06-kube-api-access-rt9kq\") pod \"service-ca-operator-cf7c776b7-jcrqk\" (UID: \"eaa018c5-5f77-486a-a95d-aa730c416c06\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jcrqk"
Sep 15 20:08:52.279587 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.279585    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/93c1a30d-050c-491d-adbc-b5e876968efd-tmp\") pod \"insights-operator-5844b856db-9cxxk\" (UID: \"93c1a30d-050c-491d-adbc-b5e876968efd\") " pod="openshift-insights/insights-operator-5844b856db-9cxxk"
Sep 15 20:08:52.280054 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:52.279610    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 15 20:08:52.280054 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:52.279627    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 15 20:08:52.280054 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:52.279680    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aea61046-fabb-4705-be56-2894d6e3ce75-samples-operator-tls podName:aea61046-fabb-4705-be56-2894d6e3ce75 nodeName:}" failed. No retries permitted until 2026-09-15 20:08:52.779661641 +0000 UTC m=+247.268126109 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/aea61046-fabb-4705-be56-2894d6e3ce75-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-s6c6z" (UID: "aea61046-fabb-4705-be56-2894d6e3ce75") : secret "samples-operator-tls" not found
Sep 15 20:08:52.280054 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:52.279697    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c085a1d9-7bd7-4769-9b58-c554276913f8-cluster-monitoring-operator-tls podName:c085a1d9-7bd7-4769-9b58-c554276913f8 nodeName:}" failed. No retries permitted until 2026-09-15 20:08:52.779688279 +0000 UTC m=+247.268152733 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c085a1d9-7bd7-4769-9b58-c554276913f8-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-wdnhk" (UID: "c085a1d9-7bd7-4769-9b58-c554276913f8") : secret "cluster-monitoring-operator-tls" not found
Sep 15 20:08:52.280054 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.279610    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93c1a30d-050c-491d-adbc-b5e876968efd-service-ca-bundle\") pod \"insights-operator-5844b856db-9cxxk\" (UID: \"93c1a30d-050c-491d-adbc-b5e876968efd\") " pod="openshift-insights/insights-operator-5844b856db-9cxxk"
Sep 15 20:08:52.280054 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.279727    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/93c1a30d-050c-491d-adbc-b5e876968efd-serving-cert\") pod \"insights-operator-5844b856db-9cxxk\" (UID: \"93c1a30d-050c-491d-adbc-b5e876968efd\") " pod="openshift-insights/insights-operator-5844b856db-9cxxk"
Sep 15 20:08:52.280054 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.279750    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eaa018c5-5f77-486a-a95d-aa730c416c06-config\") pod \"service-ca-operator-cf7c776b7-jcrqk\" (UID: \"eaa018c5-5f77-486a-a95d-aa730c416c06\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jcrqk"
Sep 15 20:08:52.280054 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.279751    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/93c1a30d-050c-491d-adbc-b5e876968efd-snapshots\") pod \"insights-operator-5844b856db-9cxxk\" (UID: \"93c1a30d-050c-491d-adbc-b5e876968efd\") " pod="openshift-insights/insights-operator-5844b856db-9cxxk"
Sep 15 20:08:52.280054 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.279813    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8r8c\" (UniqueName: \"kubernetes.io/projected/aea61046-fabb-4705-be56-2894d6e3ce75-kube-api-access-g8r8c\") pod \"cluster-samples-operator-86499c89c4-s6c6z\" (UID: \"aea61046-fabb-4705-be56-2894d6e3ce75\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s6c6z"
Sep 15 20:08:52.280512 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.280193    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93c1a30d-050c-491d-adbc-b5e876968efd-service-ca-bundle\") pod \"insights-operator-5844b856db-9cxxk\" (UID: \"93c1a30d-050c-491d-adbc-b5e876968efd\") " pod="openshift-insights/insights-operator-5844b856db-9cxxk"
Sep 15 20:08:52.280512 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.280210    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c085a1d9-7bd7-4769-9b58-c554276913f8-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-wdnhk\" (UID: \"c085a1d9-7bd7-4769-9b58-c554276913f8\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-wdnhk"
Sep 15 20:08:52.280512 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.280347    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/93c1a30d-050c-491d-adbc-b5e876968efd-tmp\") pod \"insights-operator-5844b856db-9cxxk\" (UID: \"93c1a30d-050c-491d-adbc-b5e876968efd\") " pod="openshift-insights/insights-operator-5844b856db-9cxxk"
Sep 15 20:08:52.280512 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.280355    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93c1a30d-050c-491d-adbc-b5e876968efd-trusted-ca-bundle\") pod \"insights-operator-5844b856db-9cxxk\" (UID: \"93c1a30d-050c-491d-adbc-b5e876968efd\") " pod="openshift-insights/insights-operator-5844b856db-9cxxk"
Sep 15 20:08:52.280512 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.280366    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/93c1a30d-050c-491d-adbc-b5e876968efd-snapshots\") pod \"insights-operator-5844b856db-9cxxk\" (UID: \"93c1a30d-050c-491d-adbc-b5e876968efd\") " pod="openshift-insights/insights-operator-5844b856db-9cxxk"
Sep 15 20:08:52.282665 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.282644    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eaa018c5-5f77-486a-a95d-aa730c416c06-serving-cert\") pod \"service-ca-operator-cf7c776b7-jcrqk\" (UID: \"eaa018c5-5f77-486a-a95d-aa730c416c06\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jcrqk"
Sep 15 20:08:52.282726 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.282683    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/93c1a30d-050c-491d-adbc-b5e876968efd-serving-cert\") pod \"insights-operator-5844b856db-9cxxk\" (UID: \"93c1a30d-050c-491d-adbc-b5e876968efd\") " pod="openshift-insights/insights-operator-5844b856db-9cxxk"
Sep 15 20:08:52.288094 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.288073    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rt9kq\" (UniqueName: \"kubernetes.io/projected/eaa018c5-5f77-486a-a95d-aa730c416c06-kube-api-access-rt9kq\") pod \"service-ca-operator-cf7c776b7-jcrqk\" (UID: \"eaa018c5-5f77-486a-a95d-aa730c416c06\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jcrqk"
Sep 15 20:08:52.288842 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.288815    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8gdf\" (UniqueName: \"kubernetes.io/projected/93c1a30d-050c-491d-adbc-b5e876968efd-kube-api-access-x8gdf\") pod \"insights-operator-5844b856db-9cxxk\" (UID: \"93c1a30d-050c-491d-adbc-b5e876968efd\") " pod="openshift-insights/insights-operator-5844b856db-9cxxk"
Sep 15 20:08:52.289173 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.289155    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8r8c\" (UniqueName: \"kubernetes.io/projected/aea61046-fabb-4705-be56-2894d6e3ce75-kube-api-access-g8r8c\") pod \"cluster-samples-operator-86499c89c4-s6c6z\" (UID: \"aea61046-fabb-4705-be56-2894d6e3ce75\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s6c6z"
Sep 15 20:08:52.292173 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.292144    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gb7rt\" (UniqueName: \"kubernetes.io/projected/c085a1d9-7bd7-4769-9b58-c554276913f8-kube-api-access-gb7rt\") pod \"cluster-monitoring-operator-6cd954b897-wdnhk\" (UID: \"c085a1d9-7bd7-4769-9b58-c554276913f8\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-wdnhk"
Sep 15 20:08:52.380567 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.380500    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/085814ed-8800-43b7-bc59-b4f46cf14608-service-ca-bundle\") pod \"router-default-b6b4698b-54vk2\" (UID: \"085814ed-8800-43b7-bc59-b4f46cf14608\") " pod="openshift-ingress/router-default-b6b4698b-54vk2"
Sep 15 20:08:52.380567 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.380538    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/085814ed-8800-43b7-bc59-b4f46cf14608-default-certificate\") pod \"router-default-b6b4698b-54vk2\" (UID: \"085814ed-8800-43b7-bc59-b4f46cf14608\") " pod="openshift-ingress/router-default-b6b4698b-54vk2"
Sep 15 20:08:52.380567 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.380559    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/085814ed-8800-43b7-bc59-b4f46cf14608-stats-auth\") pod \"router-default-b6b4698b-54vk2\" (UID: \"085814ed-8800-43b7-bc59-b4f46cf14608\") " pod="openshift-ingress/router-default-b6b4698b-54vk2"
Sep 15 20:08:52.380768 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.380594    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/085814ed-8800-43b7-bc59-b4f46cf14608-metrics-certs\") pod \"router-default-b6b4698b-54vk2\" (UID: \"085814ed-8800-43b7-bc59-b4f46cf14608\") " pod="openshift-ingress/router-default-b6b4698b-54vk2"
Sep 15 20:08:52.380768 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.380627    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xvnb\" (UniqueName: \"kubernetes.io/projected/085814ed-8800-43b7-bc59-b4f46cf14608-kube-api-access-4xvnb\") pod \"router-default-b6b4698b-54vk2\" (UID: \"085814ed-8800-43b7-bc59-b4f46cf14608\") " pod="openshift-ingress/router-default-b6b4698b-54vk2"
Sep 15 20:08:52.380872 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:52.380761    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 15 20:08:52.380872 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:52.380767    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/085814ed-8800-43b7-bc59-b4f46cf14608-service-ca-bundle podName:085814ed-8800-43b7-bc59-b4f46cf14608 nodeName:}" failed. No retries permitted until 2026-09-15 20:08:52.880747995 +0000 UTC m=+247.369212466 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/085814ed-8800-43b7-bc59-b4f46cf14608-service-ca-bundle") pod "router-default-b6b4698b-54vk2" (UID: "085814ed-8800-43b7-bc59-b4f46cf14608") : configmap references non-existent config key: service-ca.crt
Sep 15 20:08:52.380872 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:52.380833    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/085814ed-8800-43b7-bc59-b4f46cf14608-metrics-certs podName:085814ed-8800-43b7-bc59-b4f46cf14608 nodeName:}" failed. No retries permitted until 2026-09-15 20:08:52.88081392 +0000 UTC m=+247.369278379 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/085814ed-8800-43b7-bc59-b4f46cf14608-metrics-certs") pod "router-default-b6b4698b-54vk2" (UID: "085814ed-8800-43b7-bc59-b4f46cf14608") : secret "router-metrics-certs-default" not found
Sep 15 20:08:52.382728 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.382711    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/085814ed-8800-43b7-bc59-b4f46cf14608-default-certificate\") pod \"router-default-b6b4698b-54vk2\" (UID: \"085814ed-8800-43b7-bc59-b4f46cf14608\") " pod="openshift-ingress/router-default-b6b4698b-54vk2"
Sep 15 20:08:52.382780 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.382753    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/085814ed-8800-43b7-bc59-b4f46cf14608-stats-auth\") pod \"router-default-b6b4698b-54vk2\" (UID: \"085814ed-8800-43b7-bc59-b4f46cf14608\") " pod="openshift-ingress/router-default-b6b4698b-54vk2"
Sep 15 20:08:52.389256 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.389235    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xvnb\" (UniqueName: \"kubernetes.io/projected/085814ed-8800-43b7-bc59-b4f46cf14608-kube-api-access-4xvnb\") pod \"router-default-b6b4698b-54vk2\" (UID: \"085814ed-8800-43b7-bc59-b4f46cf14608\") " pod="openshift-ingress/router-default-b6b4698b-54vk2"
Sep 15 20:08:52.407753 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.407737    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-9cxxk"
Sep 15 20:08:52.422438 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.422419    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:08:52.440601 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.440553    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jcrqk"
Sep 15 20:08:52.483449 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.483415    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:08:52.674464 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.674334    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-9cxxk"]
Sep 15 20:08:52.676833 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:08:52.676804    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93c1a30d_050c_491d_adbc_b5e876968efd.slice/crio-373f59c662a34d9553cf83f0fbcc37ad1724d574e57a4d4ed09d9eba195cc124 WatchSource:0}: Error finding container 373f59c662a34d9553cf83f0fbcc37ad1724d574e57a4d4ed09d9eba195cc124: Status 404 returned error can't find the container with id 373f59c662a34d9553cf83f0fbcc37ad1724d574e57a4d4ed09d9eba195cc124
Sep 15 20:08:52.708250 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.708229    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-jcrqk"]
Sep 15 20:08:52.710965 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:08:52.710943    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaa018c5_5f77_486a_a95d_aa730c416c06.slice/crio-08f2e6a41ef7300f5f3ae979d931a2502227d584d256bedccab82d9b4911834f WatchSource:0}: Error finding container 08f2e6a41ef7300f5f3ae979d931a2502227d584d256bedccab82d9b4911834f: Status 404 returned error can't find the container with id 08f2e6a41ef7300f5f3ae979d931a2502227d584d256bedccab82d9b4911834f
Sep 15 20:08:52.784305 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.784279    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c085a1d9-7bd7-4769-9b58-c554276913f8-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-wdnhk\" (UID: \"c085a1d9-7bd7-4769-9b58-c554276913f8\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-wdnhk"
Sep 15 20:08:52.784408 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.784309    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/aea61046-fabb-4705-be56-2894d6e3ce75-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-s6c6z\" (UID: \"aea61046-fabb-4705-be56-2894d6e3ce75\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s6c6z"
Sep 15 20:08:52.784465 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:52.784450    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 15 20:08:52.784519 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:52.784461    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 15 20:08:52.784519 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:52.784495    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aea61046-fabb-4705-be56-2894d6e3ce75-samples-operator-tls podName:aea61046-fabb-4705-be56-2894d6e3ce75 nodeName:}" failed. No retries permitted until 2026-09-15 20:08:53.784480504 +0000 UTC m=+248.272944958 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/aea61046-fabb-4705-be56-2894d6e3ce75-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-s6c6z" (UID: "aea61046-fabb-4705-be56-2894d6e3ce75") : secret "samples-operator-tls" not found
Sep 15 20:08:52.784600 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:52.784520    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c085a1d9-7bd7-4769-9b58-c554276913f8-cluster-monitoring-operator-tls podName:c085a1d9-7bd7-4769-9b58-c554276913f8 nodeName:}" failed. No retries permitted until 2026-09-15 20:08:53.784502049 +0000 UTC m=+248.272966521 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c085a1d9-7bd7-4769-9b58-c554276913f8-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-wdnhk" (UID: "c085a1d9-7bd7-4769-9b58-c554276913f8") : secret "cluster-monitoring-operator-tls" not found
Sep 15 20:08:52.810101 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.810077    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jcrqk" event={"ID":"eaa018c5-5f77-486a-a95d-aa730c416c06","Type":"ContainerStarted","Data":"08f2e6a41ef7300f5f3ae979d931a2502227d584d256bedccab82d9b4911834f"}
Sep 15 20:08:52.811154 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.811137    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-9cxxk" event={"ID":"93c1a30d-050c-491d-adbc-b5e876968efd","Type":"ContainerStarted","Data":"373f59c662a34d9553cf83f0fbcc37ad1724d574e57a4d4ed09d9eba195cc124"}
Sep 15 20:08:52.884865 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.884839    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/085814ed-8800-43b7-bc59-b4f46cf14608-service-ca-bundle\") pod \"router-default-b6b4698b-54vk2\" (UID: \"085814ed-8800-43b7-bc59-b4f46cf14608\") " pod="openshift-ingress/router-default-b6b4698b-54vk2"
Sep 15 20:08:52.884990 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:52.884884    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/085814ed-8800-43b7-bc59-b4f46cf14608-metrics-certs\") pod \"router-default-b6b4698b-54vk2\" (UID: \"085814ed-8800-43b7-bc59-b4f46cf14608\") " pod="openshift-ingress/router-default-b6b4698b-54vk2"
Sep 15 20:08:52.885035 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:52.885010    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/085814ed-8800-43b7-bc59-b4f46cf14608-service-ca-bundle podName:085814ed-8800-43b7-bc59-b4f46cf14608 nodeName:}" failed. No retries permitted until 2026-09-15 20:08:53.884994128 +0000 UTC m=+248.373458604 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/085814ed-8800-43b7-bc59-b4f46cf14608-service-ca-bundle") pod "router-default-b6b4698b-54vk2" (UID: "085814ed-8800-43b7-bc59-b4f46cf14608") : configmap references non-existent config key: service-ca.crt
Sep 15 20:08:52.885035 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:52.885026    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 15 20:08:52.885110 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:52.885063    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/085814ed-8800-43b7-bc59-b4f46cf14608-metrics-certs podName:085814ed-8800-43b7-bc59-b4f46cf14608 nodeName:}" failed. No retries permitted until 2026-09-15 20:08:53.885051768 +0000 UTC m=+248.373516222 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/085814ed-8800-43b7-bc59-b4f46cf14608-metrics-certs") pod "router-default-b6b4698b-54vk2" (UID: "085814ed-8800-43b7-bc59-b4f46cf14608") : secret "router-metrics-certs-default" not found
Sep 15 20:08:53.792585 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:53.792548    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c085a1d9-7bd7-4769-9b58-c554276913f8-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-wdnhk\" (UID: \"c085a1d9-7bd7-4769-9b58-c554276913f8\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-wdnhk"
Sep 15 20:08:53.793066 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:53.792598    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/aea61046-fabb-4705-be56-2894d6e3ce75-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-s6c6z\" (UID: \"aea61046-fabb-4705-be56-2894d6e3ce75\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s6c6z"
Sep 15 20:08:53.793066 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:53.792684    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 15 20:08:53.793066 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:53.792771    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 15 20:08:53.793066 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:53.792795    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c085a1d9-7bd7-4769-9b58-c554276913f8-cluster-monitoring-operator-tls podName:c085a1d9-7bd7-4769-9b58-c554276913f8 nodeName:}" failed. No retries permitted until 2026-09-15 20:08:55.792773569 +0000 UTC m=+250.281238028 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c085a1d9-7bd7-4769-9b58-c554276913f8-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-wdnhk" (UID: "c085a1d9-7bd7-4769-9b58-c554276913f8") : secret "cluster-monitoring-operator-tls" not found
Sep 15 20:08:53.793066 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:53.792823    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aea61046-fabb-4705-be56-2894d6e3ce75-samples-operator-tls podName:aea61046-fabb-4705-be56-2894d6e3ce75 nodeName:}" failed. No retries permitted until 2026-09-15 20:08:55.792806723 +0000 UTC m=+250.281271194 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/aea61046-fabb-4705-be56-2894d6e3ce75-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-s6c6z" (UID: "aea61046-fabb-4705-be56-2894d6e3ce75") : secret "samples-operator-tls" not found
Sep 15 20:08:53.894147 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:53.894115    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/085814ed-8800-43b7-bc59-b4f46cf14608-service-ca-bundle\") pod \"router-default-b6b4698b-54vk2\" (UID: \"085814ed-8800-43b7-bc59-b4f46cf14608\") " pod="openshift-ingress/router-default-b6b4698b-54vk2"
Sep 15 20:08:53.894320 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:53.894177    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/085814ed-8800-43b7-bc59-b4f46cf14608-metrics-certs\") pod \"router-default-b6b4698b-54vk2\" (UID: \"085814ed-8800-43b7-bc59-b4f46cf14608\") " pod="openshift-ingress/router-default-b6b4698b-54vk2"
Sep 15 20:08:53.894320 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:53.894297    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/085814ed-8800-43b7-bc59-b4f46cf14608-service-ca-bundle podName:085814ed-8800-43b7-bc59-b4f46cf14608 nodeName:}" failed. No retries permitted until 2026-09-15 20:08:55.894261597 +0000 UTC m=+250.382726055 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/085814ed-8800-43b7-bc59-b4f46cf14608-service-ca-bundle") pod "router-default-b6b4698b-54vk2" (UID: "085814ed-8800-43b7-bc59-b4f46cf14608") : configmap references non-existent config key: service-ca.crt
Sep 15 20:08:53.894434 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:53.894333    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 15 20:08:53.894434 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:53.894370    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/085814ed-8800-43b7-bc59-b4f46cf14608-metrics-certs podName:085814ed-8800-43b7-bc59-b4f46cf14608 nodeName:}" failed. No retries permitted until 2026-09-15 20:08:55.894359072 +0000 UTC m=+250.382823530 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/085814ed-8800-43b7-bc59-b4f46cf14608-metrics-certs") pod "router-default-b6b4698b-54vk2" (UID: "085814ed-8800-43b7-bc59-b4f46cf14608") : secret "router-metrics-certs-default" not found
Sep 15 20:08:55.809694 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:55.809655    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c085a1d9-7bd7-4769-9b58-c554276913f8-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-wdnhk\" (UID: \"c085a1d9-7bd7-4769-9b58-c554276913f8\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-wdnhk"
Sep 15 20:08:55.810101 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:55.809709    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/aea61046-fabb-4705-be56-2894d6e3ce75-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-s6c6z\" (UID: \"aea61046-fabb-4705-be56-2894d6e3ce75\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s6c6z"
Sep 15 20:08:55.810101 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:55.809826    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 15 20:08:55.810101 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:55.809870    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 15 20:08:55.810101 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:55.809909    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c085a1d9-7bd7-4769-9b58-c554276913f8-cluster-monitoring-operator-tls podName:c085a1d9-7bd7-4769-9b58-c554276913f8 nodeName:}" failed. No retries permitted until 2026-09-15 20:08:59.809888287 +0000 UTC m=+254.298352749 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c085a1d9-7bd7-4769-9b58-c554276913f8-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-wdnhk" (UID: "c085a1d9-7bd7-4769-9b58-c554276913f8") : secret "cluster-monitoring-operator-tls" not found
Sep 15 20:08:55.810101 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:55.809942    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aea61046-fabb-4705-be56-2894d6e3ce75-samples-operator-tls podName:aea61046-fabb-4705-be56-2894d6e3ce75 nodeName:}" failed. No retries permitted until 2026-09-15 20:08:59.809932271 +0000 UTC m=+254.298396733 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/aea61046-fabb-4705-be56-2894d6e3ce75-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-s6c6z" (UID: "aea61046-fabb-4705-be56-2894d6e3ce75") : secret "samples-operator-tls" not found
Sep 15 20:08:55.825059 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:55.825030    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jcrqk" event={"ID":"eaa018c5-5f77-486a-a95d-aa730c416c06","Type":"ContainerStarted","Data":"d0a43d789ea01dd18910ab29776a4155ebe0cae05adaf75c29c31078481481a3"}
Sep 15 20:08:55.826566 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:55.826542    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-9cxxk" event={"ID":"93c1a30d-050c-491d-adbc-b5e876968efd","Type":"ContainerStarted","Data":"a2543a5032ebf170a0b08f991ec10d58b8a80dee0f190954727b85f08eca0b4c"}
Sep 15 20:08:55.851331 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:55.851289    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jcrqk" podStartSLOduration=1.290743418 podStartE2EDuration="3.851278031s" totalImagesPullingTime="2.560534613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:08:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:08:55.850324748 +0000 UTC m=+250.338789223" watchObservedRunningTime="2026-09-15 20:08:55.851278031 +0000 UTC m=+250.339742498"
Sep 15 20:08:55.880501 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:55.880468    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-9cxxk" podStartSLOduration=1.287335449 podStartE2EDuration="3.880459308s" totalImagesPullingTime="2.593123859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:08:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:08:55.878903967 +0000 UTC m=+250.367368456" watchObservedRunningTime="2026-09-15 20:08:55.880459308 +0000 UTC m=+250.368923783"
Sep 15 20:08:55.910991 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:55.910966    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/085814ed-8800-43b7-bc59-b4f46cf14608-service-ca-bundle\") pod \"router-default-b6b4698b-54vk2\" (UID: \"085814ed-8800-43b7-bc59-b4f46cf14608\") " pod="openshift-ingress/router-default-b6b4698b-54vk2"
Sep 15 20:08:55.911210 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:55.911194    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/085814ed-8800-43b7-bc59-b4f46cf14608-metrics-certs\") pod \"router-default-b6b4698b-54vk2\" (UID: \"085814ed-8800-43b7-bc59-b4f46cf14608\") " pod="openshift-ingress/router-default-b6b4698b-54vk2"
Sep 15 20:08:55.911491 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:55.911475    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 15 20:08:55.911617 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:55.911609    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/085814ed-8800-43b7-bc59-b4f46cf14608-metrics-certs podName:085814ed-8800-43b7-bc59-b4f46cf14608 nodeName:}" failed. No retries permitted until 2026-09-15 20:08:59.911591477 +0000 UTC m=+254.400055954 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/085814ed-8800-43b7-bc59-b4f46cf14608-metrics-certs") pod "router-default-b6b4698b-54vk2" (UID: "085814ed-8800-43b7-bc59-b4f46cf14608") : secret "router-metrics-certs-default" not found
Sep 15 20:08:55.911766 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:55.911721    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/085814ed-8800-43b7-bc59-b4f46cf14608-service-ca-bundle podName:085814ed-8800-43b7-bc59-b4f46cf14608 nodeName:}" failed. No retries permitted until 2026-09-15 20:08:59.911700912 +0000 UTC m=+254.400165393 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/085814ed-8800-43b7-bc59-b4f46cf14608-service-ca-bundle") pod "router-default-b6b4698b-54vk2" (UID: "085814ed-8800-43b7-bc59-b4f46cf14608") : configmap references non-existent config key: service-ca.crt
Sep 15 20:08:56.118855 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:56.118773    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ns6zw"]
Sep 15 20:08:56.119580 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:56.119555    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:08:56.121138 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:56.121122    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ns6zw"
Sep 15 20:08:56.123387 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:56.123369    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 15 20:08:56.123863 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:56.123840    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 15 20:08:56.124305 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:56.124287    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8swwd\""
Sep 15 20:08:56.140786 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:56.140766    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ns6zw"]
Sep 15 20:08:56.202642 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:56.202616    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:08:56.214412 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:56.214390    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ns6zw\" (UID: \"d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f\") " pod="openshift-insights/insights-runtime-extractor-ns6zw"
Sep 15 20:08:56.214550 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:56.214419    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ns6zw\" (UID: \"d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f\") " pod="openshift-insights/insights-runtime-extractor-ns6zw"
Sep 15 20:08:56.214622 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:56.214600    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-crio-socket\") pod \"insights-runtime-extractor-ns6zw\" (UID: \"d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f\") " pod="openshift-insights/insights-runtime-extractor-ns6zw"
Sep 15 20:08:56.214682 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:56.214665    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-data-volume\") pod \"insights-runtime-extractor-ns6zw\" (UID: \"d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f\") " pod="openshift-insights/insights-runtime-extractor-ns6zw"
Sep 15 20:08:56.214729 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:56.214692    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkp4m\" (UniqueName: \"kubernetes.io/projected/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-kube-api-access-dkp4m\") pod \"insights-runtime-extractor-ns6zw\" (UID: \"d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f\") " pod="openshift-insights/insights-runtime-extractor-ns6zw"
Sep 15 20:08:56.315237 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:56.315216    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-crio-socket\") pod \"insights-runtime-extractor-ns6zw\" (UID: \"d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f\") " pod="openshift-insights/insights-runtime-extractor-ns6zw"
Sep 15 20:08:56.315356 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:56.315256    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-data-volume\") pod \"insights-runtime-extractor-ns6zw\" (UID: \"d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f\") " pod="openshift-insights/insights-runtime-extractor-ns6zw"
Sep 15 20:08:56.315356 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:56.315275    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkp4m\" (UniqueName: \"kubernetes.io/projected/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-kube-api-access-dkp4m\") pod \"insights-runtime-extractor-ns6zw\" (UID: \"d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f\") " pod="openshift-insights/insights-runtime-extractor-ns6zw"
Sep 15 20:08:56.315356 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:56.315337    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-crio-socket\") pod \"insights-runtime-extractor-ns6zw\" (UID: \"d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f\") " pod="openshift-insights/insights-runtime-extractor-ns6zw"
Sep 15 20:08:56.315553 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:56.315443    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ns6zw\" (UID: \"d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f\") " pod="openshift-insights/insights-runtime-extractor-ns6zw"
Sep 15 20:08:56.315553 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:56.315495    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ns6zw\" (UID: \"d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f\") " pod="openshift-insights/insights-runtime-extractor-ns6zw"
Sep 15 20:08:56.315553 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:56.315516    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-data-volume\") pod \"insights-runtime-extractor-ns6zw\" (UID: \"d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f\") " pod="openshift-insights/insights-runtime-extractor-ns6zw"
Sep 15 20:08:56.315553 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:56.315523    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 20:08:56.315721 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:56.315577    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-insights-runtime-extractor-tls podName:d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f nodeName:}" failed. No retries permitted until 2026-09-15 20:08:56.815559509 +0000 UTC m=+251.304023996 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ns6zw" (UID: "d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f") : secret "insights-runtime-extractor-tls" not found
Sep 15 20:08:56.315721 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:56.315583    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 20:08:56.315721 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:56.315632    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-kube-rbac-proxy-cm podName:d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f nodeName:}" failed. No retries permitted until 2026-09-15 20:08:56.815615754 +0000 UTC m=+251.304080226 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ns6zw" (UID: "d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f") : configmap "kube-rbac-proxy" not found
Sep 15 20:08:56.325254 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:56.325235    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkp4m\" (UniqueName: \"kubernetes.io/projected/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-kube-api-access-dkp4m\") pod \"insights-runtime-extractor-ns6zw\" (UID: \"d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f\") " pod="openshift-insights/insights-runtime-extractor-ns6zw"
Sep 15 20:08:56.819417 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:56.819378    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ns6zw\" (UID: \"d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f\") " pod="openshift-insights/insights-runtime-extractor-ns6zw"
Sep 15 20:08:56.819804 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:56.819431    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ns6zw\" (UID: \"d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f\") " pod="openshift-insights/insights-runtime-extractor-ns6zw"
Sep 15 20:08:56.819804 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:56.819531    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 20:08:56.819804 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:56.819568    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 20:08:56.819804 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:56.819594    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-insights-runtime-extractor-tls podName:d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f nodeName:}" failed. No retries permitted until 2026-09-15 20:08:57.819579509 +0000 UTC m=+252.308043963 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ns6zw" (UID: "d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f") : secret "insights-runtime-extractor-tls" not found
Sep 15 20:08:56.819804 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:56.819618    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-kube-rbac-proxy-cm podName:d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f nodeName:}" failed. No retries permitted until 2026-09-15 20:08:57.819605533 +0000 UTC m=+252.308070007 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ns6zw" (UID: "d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f") : configmap "kube-rbac-proxy" not found
Sep 15 20:08:56.940148 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:56.940117    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:08:57.063464 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:57.063427    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:08:57.202180 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:57.202114    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:08:57.726333 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:57.726296    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-metrics-certs\") pod \"network-metrics-daemon-59mfs\" (UID: \"3b8d9342-6574-4d0d-a326-0299eb3b4c5d\") " pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:08:57.726614 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:57.726469    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 15 20:08:57.726614 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:57.726553    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-metrics-certs podName:3b8d9342-6574-4d0d-a326-0299eb3b4c5d nodeName:}" failed. No retries permitted until 2026-09-15 20:10:59.726530897 +0000 UTC m=+374.214995369 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-metrics-certs") pod "network-metrics-daemon-59mfs" (UID: "3b8d9342-6574-4d0d-a326-0299eb3b4c5d") : secret "metrics-daemon-secret" not found
Sep 15 20:08:57.827696 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:57.827659    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ns6zw\" (UID: \"d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f\") " pod="openshift-insights/insights-runtime-extractor-ns6zw"
Sep 15 20:08:57.828063 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:57.827700    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ns6zw\" (UID: \"d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f\") " pod="openshift-insights/insights-runtime-extractor-ns6zw"
Sep 15 20:08:57.828063 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:57.827794    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 20:08:57.828063 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:57.827801    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 20:08:57.828063 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:57.827855    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-insights-runtime-extractor-tls podName:d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f nodeName:}" failed. No retries permitted until 2026-09-15 20:08:59.827841297 +0000 UTC m=+254.316305754 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ns6zw" (UID: "d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f") : secret "insights-runtime-extractor-tls" not found
Sep 15 20:08:57.828063 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:57.827870    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-kube-rbac-proxy-cm podName:d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f nodeName:}" failed. No retries permitted until 2026-09-15 20:08:59.827864372 +0000 UTC m=+254.316328832 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ns6zw" (UID: "d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f") : configmap "kube-rbac-proxy" not found
Sep 15 20:08:57.843102 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:57.843073    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:08:57.985220 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:57.985140    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:08:58.059420 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:58.059397    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-s8ddr_6cd01117-433d-4273-a167-01344ad4f018/dns-node-resolver/0.log"
Sep 15 20:08:58.107035 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:58.107000    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:08:58.795507 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:58.795475    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:08:58.896176 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:58.896139    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:08:59.056448 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:59.056378    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qk9ld_48d7dc57-6de6-4a72-8da3-156c90d7a8ff/node-ca/0.log"
Sep 15 20:08:59.656955 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:59.656909    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-49mwc_7a1d40a8-4253-43b6-b7c8-d278b7649dcd/serve-healthcheck-canary/0.log"
Sep 15 20:08:59.844935 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:59.844892    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c085a1d9-7bd7-4769-9b58-c554276913f8-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-wdnhk\" (UID: \"c085a1d9-7bd7-4769-9b58-c554276913f8\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-wdnhk"
Sep 15 20:08:59.844935 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:59.844937    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/aea61046-fabb-4705-be56-2894d6e3ce75-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-s6c6z\" (UID: \"aea61046-fabb-4705-be56-2894d6e3ce75\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s6c6z"
Sep 15 20:08:59.845129 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:59.845042    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 15 20:08:59.845129 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:59.845073    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ns6zw\" (UID: \"d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f\") " pod="openshift-insights/insights-runtime-extractor-ns6zw"
Sep 15 20:08:59.845129 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:59.845044    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 15 20:08:59.845129 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:59.845102    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c085a1d9-7bd7-4769-9b58-c554276913f8-cluster-monitoring-operator-tls podName:c085a1d9-7bd7-4769-9b58-c554276913f8 nodeName:}" failed. No retries permitted until 2026-09-15 20:09:07.845081281 +0000 UTC m=+262.333545735 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c085a1d9-7bd7-4769-9b58-c554276913f8-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-wdnhk" (UID: "c085a1d9-7bd7-4769-9b58-c554276913f8") : secret "cluster-monitoring-operator-tls" not found
Sep 15 20:08:59.845129 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:59.845118    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 20:08:59.845129 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:59.845129    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aea61046-fabb-4705-be56-2894d6e3ce75-samples-operator-tls podName:aea61046-fabb-4705-be56-2894d6e3ce75 nodeName:}" failed. No retries permitted until 2026-09-15 20:09:07.845116936 +0000 UTC m=+262.333581441 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/aea61046-fabb-4705-be56-2894d6e3ce75-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-s6c6z" (UID: "aea61046-fabb-4705-be56-2894d6e3ce75") : secret "samples-operator-tls" not found
Sep 15 20:08:59.845357 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:59.845153    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-insights-runtime-extractor-tls podName:d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f nodeName:}" failed. No retries permitted until 2026-09-15 20:09:03.845142109 +0000 UTC m=+258.333606580 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ns6zw" (UID: "d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f") : secret "insights-runtime-extractor-tls" not found
Sep 15 20:08:59.845357 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:59.845165    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ns6zw\" (UID: \"d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f\") " pod="openshift-insights/insights-runtime-extractor-ns6zw"
Sep 15 20:08:59.845357 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:59.845237    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 20:08:59.845357 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:59.845264    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-kube-rbac-proxy-cm podName:d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f nodeName:}" failed. No retries permitted until 2026-09-15 20:09:03.845257201 +0000 UTC m=+258.333721656 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ns6zw" (UID: "d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f") : configmap "kube-rbac-proxy" not found
Sep 15 20:08:59.945526 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:59.945456    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/085814ed-8800-43b7-bc59-b4f46cf14608-service-ca-bundle\") pod \"router-default-b6b4698b-54vk2\" (UID: \"085814ed-8800-43b7-bc59-b4f46cf14608\") " pod="openshift-ingress/router-default-b6b4698b-54vk2"
Sep 15 20:08:59.945526 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:59.945517    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/085814ed-8800-43b7-bc59-b4f46cf14608-metrics-certs\") pod \"router-default-b6b4698b-54vk2\" (UID: \"085814ed-8800-43b7-bc59-b4f46cf14608\") " pod="openshift-ingress/router-default-b6b4698b-54vk2"
Sep 15 20:08:59.945910 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:59.945601    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/085814ed-8800-43b7-bc59-b4f46cf14608-service-ca-bundle podName:085814ed-8800-43b7-bc59-b4f46cf14608 nodeName:}" failed. No retries permitted until 2026-09-15 20:09:07.945582871 +0000 UTC m=+262.434047342 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/085814ed-8800-43b7-bc59-b4f46cf14608-service-ca-bundle") pod "router-default-b6b4698b-54vk2" (UID: "085814ed-8800-43b7-bc59-b4f46cf14608") : configmap references non-existent config key: service-ca.crt
Sep 15 20:08:59.945910 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:59.945650    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 15 20:08:59.945910 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:08:59.945693    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/085814ed-8800-43b7-bc59-b4f46cf14608-metrics-certs podName:085814ed-8800-43b7-bc59-b4f46cf14608 nodeName:}" failed. No retries permitted until 2026-09-15 20:09:07.945680741 +0000 UTC m=+262.434145201 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/085814ed-8800-43b7-bc59-b4f46cf14608-metrics-certs") pod "router-default-b6b4698b-54vk2" (UID: "085814ed-8800-43b7-bc59-b4f46cf14608") : secret "router-metrics-certs-default" not found
Sep 15 20:08:59.945910 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:08:59.945740    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:00.046516 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:00.046484    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:00.123000 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:00.122966    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-vcwks"]
Sep 15 20:09:00.125355 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:00.125342    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-vcwks"
Sep 15 20:09:00.128132 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:00.128113    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-jzkf9\""
Sep 15 20:09:00.128132 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:00.128113    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 15 20:09:00.128604 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:00.128582    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 15 20:09:00.128667 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:00.128622    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 15 20:09:00.128724 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:00.128669    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 15 20:09:00.141636 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:00.141615    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-vcwks"]
Sep 15 20:09:00.163627 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:00.163595    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:00.229662 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:00.229633    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:00.247772 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:00.247754    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a598d94c-5576-4dbf-8686-c9f90cd1e552-signing-key\") pod \"service-ca-568869fdc5-vcwks\" (UID: \"a598d94c-5576-4dbf-8686-c9f90cd1e552\") " pod="openshift-service-ca/service-ca-568869fdc5-vcwks"
Sep 15 20:09:00.247832 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:00.247777    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sggxf\" (UniqueName: \"kubernetes.io/projected/a598d94c-5576-4dbf-8686-c9f90cd1e552-kube-api-access-sggxf\") pod \"service-ca-568869fdc5-vcwks\" (UID: \"a598d94c-5576-4dbf-8686-c9f90cd1e552\") " pod="openshift-service-ca/service-ca-568869fdc5-vcwks"
Sep 15 20:09:00.247903 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:00.247887    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a598d94c-5576-4dbf-8686-c9f90cd1e552-signing-cabundle\") pod \"service-ca-568869fdc5-vcwks\" (UID: \"a598d94c-5576-4dbf-8686-c9f90cd1e552\") " pod="openshift-service-ca/service-ca-568869fdc5-vcwks"
Sep 15 20:09:00.349216 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:00.349190    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a598d94c-5576-4dbf-8686-c9f90cd1e552-signing-cabundle\") pod \"service-ca-568869fdc5-vcwks\" (UID: \"a598d94c-5576-4dbf-8686-c9f90cd1e552\") " pod="openshift-service-ca/service-ca-568869fdc5-vcwks"
Sep 15 20:09:00.349336 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:00.349268    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a598d94c-5576-4dbf-8686-c9f90cd1e552-signing-key\") pod \"service-ca-568869fdc5-vcwks\" (UID: \"a598d94c-5576-4dbf-8686-c9f90cd1e552\") " pod="openshift-service-ca/service-ca-568869fdc5-vcwks"
Sep 15 20:09:00.349336 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:00.349285    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sggxf\" (UniqueName: \"kubernetes.io/projected/a598d94c-5576-4dbf-8686-c9f90cd1e552-kube-api-access-sggxf\") pod \"service-ca-568869fdc5-vcwks\" (UID: \"a598d94c-5576-4dbf-8686-c9f90cd1e552\") " pod="openshift-service-ca/service-ca-568869fdc5-vcwks"
Sep 15 20:09:00.349832 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:00.349809    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a598d94c-5576-4dbf-8686-c9f90cd1e552-signing-cabundle\") pod \"service-ca-568869fdc5-vcwks\" (UID: \"a598d94c-5576-4dbf-8686-c9f90cd1e552\") " pod="openshift-service-ca/service-ca-568869fdc5-vcwks"
Sep 15 20:09:00.351703 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:00.351678    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a598d94c-5576-4dbf-8686-c9f90cd1e552-signing-key\") pod \"service-ca-568869fdc5-vcwks\" (UID: \"a598d94c-5576-4dbf-8686-c9f90cd1e552\") " pod="openshift-service-ca/service-ca-568869fdc5-vcwks"
Sep 15 20:09:00.358628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:00.358611    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sggxf\" (UniqueName: \"kubernetes.io/projected/a598d94c-5576-4dbf-8686-c9f90cd1e552-kube-api-access-sggxf\") pod \"service-ca-568869fdc5-vcwks\" (UID: \"a598d94c-5576-4dbf-8686-c9f90cd1e552\") " pod="openshift-service-ca/service-ca-568869fdc5-vcwks"
Sep 15 20:09:00.433869 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:00.433845    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-vcwks"
Sep 15 20:09:00.702395 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:00.702366    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-vcwks"]
Sep 15 20:09:00.705705 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:09:00.705681    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda598d94c_5576_4dbf_8686_c9f90cd1e552.slice/crio-10272b469d17d89471dd0b19c9e0ada751081261fd4d0263a305aef702c537f2 WatchSource:0}: Error finding container 10272b469d17d89471dd0b19c9e0ada751081261fd4d0263a305aef702c537f2: Status 404 returned error can't find the container with id 10272b469d17d89471dd0b19c9e0ada751081261fd4d0263a305aef702c537f2
Sep 15 20:09:00.840306 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:00.840272    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-vcwks" event={"ID":"a598d94c-5576-4dbf-8686-c9f90cd1e552","Type":"ContainerStarted","Data":"55b2cd7168e2843f1227fe1b300fb7765e707ecad540259110826deac1b9d3e7"}
Sep 15 20:09:00.840483 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:00.840323    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-vcwks" event={"ID":"a598d94c-5576-4dbf-8686-c9f90cd1e552","Type":"ContainerStarted","Data":"10272b469d17d89471dd0b19c9e0ada751081261fd4d0263a305aef702c537f2"}
Sep 15 20:09:00.865099 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:00.865051    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-vcwks" podStartSLOduration=0.865036292 podStartE2EDuration="865.036292ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:09:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:09:00.863793192 +0000 UTC m=+255.352257668" watchObservedRunningTime="2026-09-15 20:09:00.865036292 +0000 UTC m=+255.353500767"
Sep 15 20:09:01.151461 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:01.151375    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:02.351586 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:02.351551    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:02.602396 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:02.602357    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:03.323772 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:03.323739    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:03.397489 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:03.397461    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:03.880138 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:03.880099    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ns6zw\" (UID: \"d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f\") " pod="openshift-insights/insights-runtime-extractor-ns6zw"
Sep 15 20:09:03.880331 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:03.880145    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ns6zw\" (UID: \"d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f\") " pod="openshift-insights/insights-runtime-extractor-ns6zw"
Sep 15 20:09:03.880331 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:09:03.880254    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 20:09:03.880331 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:09:03.880287    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 20:09:03.880331 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:09:03.880313    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-insights-runtime-extractor-tls podName:d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f nodeName:}" failed. No retries permitted until 2026-09-15 20:09:11.880298619 +0000 UTC m=+266.368763078 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ns6zw" (UID: "d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f") : secret "insights-runtime-extractor-tls" not found
Sep 15 20:09:03.880331 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:09:03.880333    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-kube-rbac-proxy-cm podName:d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f nodeName:}" failed. No retries permitted until 2026-09-15 20:09:11.880320507 +0000 UTC m=+266.368784965 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ns6zw" (UID: "d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f") : configmap "kube-rbac-proxy" not found
Sep 15 20:09:04.054382 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:04.054351    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:04.446185 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:04.446151    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:04.653858 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:04.653826    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:04.871729 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:04.871697    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:05.052717 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:05.052679    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:05.155339 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:05.155263    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:05.965867 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:05.965842    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:06.262744 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:06.262667    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:06.364848 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:06.364781    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:06.472169 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:06.472139    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:06.658183 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:06.658115    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:06.846414 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:06.846388    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:07.738994 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:07.738957    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:07.913971 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:07.913915    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c085a1d9-7bd7-4769-9b58-c554276913f8-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-wdnhk\" (UID: \"c085a1d9-7bd7-4769-9b58-c554276913f8\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-wdnhk"
Sep 15 20:09:07.913971 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:07.913971    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/aea61046-fabb-4705-be56-2894d6e3ce75-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-s6c6z\" (UID: \"aea61046-fabb-4705-be56-2894d6e3ce75\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s6c6z"
Sep 15 20:09:07.914200 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:09:07.914064    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 15 20:09:07.914200 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:09:07.914126    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c085a1d9-7bd7-4769-9b58-c554276913f8-cluster-monitoring-operator-tls podName:c085a1d9-7bd7-4769-9b58-c554276913f8 nodeName:}" failed. No retries permitted until 2026-09-15 20:09:23.914108994 +0000 UTC m=+278.402573447 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c085a1d9-7bd7-4769-9b58-c554276913f8-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-wdnhk" (UID: "c085a1d9-7bd7-4769-9b58-c554276913f8") : secret "cluster-monitoring-operator-tls" not found
Sep 15 20:09:07.916291 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:07.916274    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/aea61046-fabb-4705-be56-2894d6e3ce75-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-s6c6z\" (UID: \"aea61046-fabb-4705-be56-2894d6e3ce75\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s6c6z"
Sep 15 20:09:08.014650 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:08.014576    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/085814ed-8800-43b7-bc59-b4f46cf14608-service-ca-bundle\") pod \"router-default-b6b4698b-54vk2\" (UID: \"085814ed-8800-43b7-bc59-b4f46cf14608\") " pod="openshift-ingress/router-default-b6b4698b-54vk2"
Sep 15 20:09:08.014787 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:08.014670    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/085814ed-8800-43b7-bc59-b4f46cf14608-metrics-certs\") pod \"router-default-b6b4698b-54vk2\" (UID: \"085814ed-8800-43b7-bc59-b4f46cf14608\") " pod="openshift-ingress/router-default-b6b4698b-54vk2"
Sep 15 20:09:08.014787 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:09:08.014731    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/085814ed-8800-43b7-bc59-b4f46cf14608-service-ca-bundle podName:085814ed-8800-43b7-bc59-b4f46cf14608 nodeName:}" failed. No retries permitted until 2026-09-15 20:09:24.014715208 +0000 UTC m=+278.503179667 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/085814ed-8800-43b7-bc59-b4f46cf14608-service-ca-bundle") pod "router-default-b6b4698b-54vk2" (UID: "085814ed-8800-43b7-bc59-b4f46cf14608") : configmap references non-existent config key: service-ca.crt
Sep 15 20:09:08.017022 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:08.017002    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/085814ed-8800-43b7-bc59-b4f46cf14608-metrics-certs\") pod \"router-default-b6b4698b-54vk2\" (UID: \"085814ed-8800-43b7-bc59-b4f46cf14608\") " pod="openshift-ingress/router-default-b6b4698b-54vk2"
Sep 15 20:09:08.035475 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:08.035453    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s6c6z"
Sep 15 20:09:08.045550 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:08.045529    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:08.252781 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:08.252712    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:08.310811 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:08.310747    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s6c6z"]
Sep 15 20:09:08.671031 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:08.670959    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:08.841686 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:08.841655    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:08.869342 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:08.869312    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s6c6z" event={"ID":"aea61046-fabb-4705-be56-2894d6e3ce75","Type":"ContainerStarted","Data":"e3d1d5449a129c1d4eefce0d7265299c72b30d193140d312403f7597be1db191"}
Sep 15 20:09:10.877400 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:10.877371    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s6c6z" event={"ID":"aea61046-fabb-4705-be56-2894d6e3ce75","Type":"ContainerStarted","Data":"5f866e51dccc0f18016a8f08cfd19d6038d2ef9b1cd157b1d148dbc28cdf6cb5"}
Sep 15 20:09:10.877830 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:10.877421    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s6c6z" event={"ID":"aea61046-fabb-4705-be56-2894d6e3ce75","Type":"ContainerStarted","Data":"a3f7e75cab5f7e44059e5af7a22b7a7a512e04d90feec59bae55eada0e38e850"}
Sep 15 20:09:10.902076 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:10.901985    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s6c6z" podStartSLOduration=16.61253576 podStartE2EDuration="18.901969527s" totalImagesPullingTime="2.289433767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:08:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:09:10.901162633 +0000 UTC m=+265.389627111" watchObservedRunningTime="2026-09-15 20:09:10.901969527 +0000 UTC m=+265.390434004"
Sep 15 20:09:11.432647 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:11.432617    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:11.506838 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:11.506783    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:11.859823 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:11.856657    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:11.947992 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:11.947947    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ns6zw\" (UID: \"d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f\") " pod="openshift-insights/insights-runtime-extractor-ns6zw"
Sep 15 20:09:11.948401 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:11.948010    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ns6zw\" (UID: \"d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f\") " pod="openshift-insights/insights-runtime-extractor-ns6zw"
Sep 15 20:09:11.948401 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:09:11.948063    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 20:09:11.948401 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:09:11.948132    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-insights-runtime-extractor-tls podName:d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f nodeName:}" failed. No retries permitted until 2026-09-15 20:09:27.948112566 +0000 UTC m=+282.436577027 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ns6zw" (UID: "d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f") : secret "insights-runtime-extractor-tls" not found
Sep 15 20:09:11.948577 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:11.948558    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ns6zw\" (UID: \"d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f\") " pod="openshift-insights/insights-runtime-extractor-ns6zw"
Sep 15 20:09:12.441402 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:12.441373    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:12.649192 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:12.649158    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:12.718982 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:12.718953    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:12.863788 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:12.863758    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:13.051652 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:13.051589    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:20.145641 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:20.145605    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:20.246174 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:20.246148    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:20.306972 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:20.306946    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:20.449706 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:20.449655    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:20.551973 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:20.551941    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:20.651594 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:20.651568    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:20.834181 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:20.834150    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:20.908020 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:20.907992    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:21.224371 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:21.224336    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:21.334512 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:21.334457    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:21.428905 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:21.428876    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:21.837062 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:21.837029    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:21.937070 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:21.937040    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:22.039884 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:22.039859    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:22.657045 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:22.657004    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:22.717940 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:22.717901    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:23.935356 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:23.935322    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c085a1d9-7bd7-4769-9b58-c554276913f8-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-wdnhk\" (UID: \"c085a1d9-7bd7-4769-9b58-c554276913f8\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-wdnhk"
Sep 15 20:09:23.937616 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:23.937594    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c085a1d9-7bd7-4769-9b58-c554276913f8-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-wdnhk\" (UID: \"c085a1d9-7bd7-4769-9b58-c554276913f8\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-wdnhk"
Sep 15 20:09:24.035809 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:24.035783    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/085814ed-8800-43b7-bc59-b4f46cf14608-service-ca-bundle\") pod \"router-default-b6b4698b-54vk2\" (UID: \"085814ed-8800-43b7-bc59-b4f46cf14608\") " pod="openshift-ingress/router-default-b6b4698b-54vk2"
Sep 15 20:09:24.036311 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:24.036292    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/085814ed-8800-43b7-bc59-b4f46cf14608-service-ca-bundle\") pod \"router-default-b6b4698b-54vk2\" (UID: \"085814ed-8800-43b7-bc59-b4f46cf14608\") " pod="openshift-ingress/router-default-b6b4698b-54vk2"
Sep 15 20:09:24.227081 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:24.227056    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-wdnhk"
Sep 15 20:09:24.315238 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:24.315214    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-b6b4698b-54vk2"
Sep 15 20:09:24.512154 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:24.512084    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-wdnhk"]
Sep 15 20:09:24.514520 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:09:24.514491    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc085a1d9_7bd7_4769_9b58_c554276913f8.slice/crio-cc2b5b836f6422dfffb3fee53db331549373bfd9422279100de93eff5fd6ebec WatchSource:0}: Error finding container cc2b5b836f6422dfffb3fee53db331549373bfd9422279100de93eff5fd6ebec: Status 404 returned error can't find the container with id cc2b5b836f6422dfffb3fee53db331549373bfd9422279100de93eff5fd6ebec
Sep 15 20:09:24.600500 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:24.600476    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-b6b4698b-54vk2"]
Sep 15 20:09:24.603088 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:09:24.603056    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod085814ed_8800_43b7_bc59_b4f46cf14608.slice/crio-eba2cb4dfecf5aec8fc6ade90818762fa13d396685ae5089873dd4bbb8348620 WatchSource:0}: Error finding container eba2cb4dfecf5aec8fc6ade90818762fa13d396685ae5089873dd4bbb8348620: Status 404 returned error can't find the container with id eba2cb4dfecf5aec8fc6ade90818762fa13d396685ae5089873dd4bbb8348620
Sep 15 20:09:24.916703 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:24.916624    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-b6b4698b-54vk2" event={"ID":"085814ed-8800-43b7-bc59-b4f46cf14608","Type":"ContainerStarted","Data":"cef1908ace5878bcafad01fa94776e56a70d9917537d1e5d32ffd773835c7afc"}
Sep 15 20:09:24.916703 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:24.916683    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-b6b4698b-54vk2" event={"ID":"085814ed-8800-43b7-bc59-b4f46cf14608","Type":"ContainerStarted","Data":"eba2cb4dfecf5aec8fc6ade90818762fa13d396685ae5089873dd4bbb8348620"}
Sep 15 20:09:24.917821 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:24.917797    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-wdnhk" event={"ID":"c085a1d9-7bd7-4769-9b58-c554276913f8","Type":"ContainerStarted","Data":"cc2b5b836f6422dfffb3fee53db331549373bfd9422279100de93eff5fd6ebec"}
Sep 15 20:09:24.945844 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:24.945776    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-b6b4698b-54vk2" podStartSLOduration=32.945755239 podStartE2EDuration="32.945755239s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:08:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:09:24.944737855 +0000 UTC m=+279.433202334" watchObservedRunningTime="2026-09-15 20:09:24.945755239 +0000 UTC m=+279.434219716"
Sep 15 20:09:25.315605 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:25.315571    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-b6b4698b-54vk2"
Sep 15 20:09:25.317989 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:25.317968    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-b6b4698b-54vk2"
Sep 15 20:09:25.600636 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:09:25.600554    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-557757c5cd-9l2fg" podUID="9cb71e8f-b673-49d6-bce3-2b205b80b22f"
Sep 15 20:09:25.600636 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:09:25.600552    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-qg2bs" podUID="0c65a532-9608-4066-9b1a-156727664921"
Sep 15 20:09:25.600636 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:09:25.600556    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kszh" podUID="fcb6cd63-75f0-4efd-a878-fb7236f4109b"
Sep 15 20:09:25.921423 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:25.921346    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kszh"
Sep 15 20:09:25.921580 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:25.921347    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qg2bs"
Sep 15 20:09:25.921580 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:25.921347    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:09:25.921687 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:25.921630    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-b6b4698b-54vk2"
Sep 15 20:09:25.923021 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:25.923002    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-b6b4698b-54vk2"
Sep 15 20:09:26.635708 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:26.635636    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:26.738863 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:26.738834    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:26.819942 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:26.819898    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-87qsf"]
Sep 15 20:09:26.821731 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:26.821714    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-87qsf"
Sep 15 20:09:26.823960 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:26.823940    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 15 20:09:26.824048 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:26.823965    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-jkrs8\""
Sep 15 20:09:26.834295 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:26.834275    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-87qsf"]
Sep 15 20:09:26.926759 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:26.926692    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-wdnhk" event={"ID":"c085a1d9-7bd7-4769-9b58-c554276913f8","Type":"ContainerStarted","Data":"943ee15975a19684d4a99827b268aef6c6100ae4ad97bb86804ca107e208327e"}
Sep 15 20:09:26.947399 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:26.947346    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-wdnhk" podStartSLOduration=33.163153445 podStartE2EDuration="34.947326839s" totalImagesPullingTime="1.784173394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:08:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:09:26.946290172 +0000 UTC m=+281.434754649" watchObservedRunningTime="2026-09-15 20:09:26.947326839 +0000 UTC m=+281.435791314"
Sep 15 20:09:26.959547 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:26.959522    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/33eaf6a7-ab22-48a4-9502-3fc7943c9c27-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-87qsf\" (UID: \"33eaf6a7-ab22-48a4-9502-3fc7943c9c27\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-87qsf"
Sep 15 20:09:27.060397 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:27.060363    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/33eaf6a7-ab22-48a4-9502-3fc7943c9c27-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-87qsf\" (UID: \"33eaf6a7-ab22-48a4-9502-3fc7943c9c27\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-87qsf"
Sep 15 20:09:27.062802 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:27.062780    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/33eaf6a7-ab22-48a4-9502-3fc7943c9c27-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-87qsf\" (UID: \"33eaf6a7-ab22-48a4-9502-3fc7943c9c27\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-87qsf"
Sep 15 20:09:27.131103 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:27.131079    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-87qsf"
Sep 15 20:09:27.183240 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:27.182993    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:27.248606 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:27.248577    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:27.397012 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:27.396984    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-87qsf"]
Sep 15 20:09:27.399775 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:09:27.399751    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33eaf6a7_ab22_48a4_9502_3fc7943c9c27.slice/crio-6af75fb802ad762249e7c1cd98ed4ebc3bb0920eb07e61a80b5617bc1503a45b WatchSource:0}: Error finding container 6af75fb802ad762249e7c1cd98ed4ebc3bb0920eb07e61a80b5617bc1503a45b: Status 404 returned error can't find the container with id 6af75fb802ad762249e7c1cd98ed4ebc3bb0920eb07e61a80b5617bc1503a45b
Sep 15 20:09:27.933645 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:27.933612    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-87qsf" event={"ID":"33eaf6a7-ab22-48a4-9502-3fc7943c9c27","Type":"ContainerStarted","Data":"6af75fb802ad762249e7c1cd98ed4ebc3bb0920eb07e61a80b5617bc1503a45b"}
Sep 15 20:09:27.967299 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:27.967266    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ns6zw\" (UID: \"d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f\") " pod="openshift-insights/insights-runtime-extractor-ns6zw"
Sep 15 20:09:27.969476 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:27.969455    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ns6zw\" (UID: \"d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f\") " pod="openshift-insights/insights-runtime-extractor-ns6zw"
Sep 15 20:09:28.230551 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:28.230520    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ns6zw"
Sep 15 20:09:28.502984 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:28.502887    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ns6zw"]
Sep 15 20:09:28.593565 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:09:28.593527    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2cf9195_fe5f_41ef_8d06_65de3e0e4e3f.slice/crio-337d1b55cb2158c48529fc54c932b297f4fff9e7832be690e0c56dff7c13ba07 WatchSource:0}: Error finding container 337d1b55cb2158c48529fc54c932b297f4fff9e7832be690e0c56dff7c13ba07: Status 404 returned error can't find the container with id 337d1b55cb2158c48529fc54c932b297f4fff9e7832be690e0c56dff7c13ba07
Sep 15 20:09:28.774901 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:28.774829    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0c65a532-9608-4066-9b1a-156727664921-metrics-tls\") pod \"dns-default-qg2bs\" (UID: \"0c65a532-9608-4066-9b1a-156727664921\") " pod="openshift-dns/dns-default-qg2bs"
Sep 15 20:09:28.774901 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:28.774876    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-tls\") pod \"image-registry-557757c5cd-9l2fg\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") " pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:09:28.775070 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:28.774908    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fcb6cd63-75f0-4efd-a878-fb7236f4109b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-2kszh\" (UID: \"fcb6cd63-75f0-4efd-a878-fb7236f4109b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kszh"
Sep 15 20:09:28.777244 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:28.777216    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-tls\") pod \"image-registry-557757c5cd-9l2fg\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") " pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:09:28.777359 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:28.777219    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fcb6cd63-75f0-4efd-a878-fb7236f4109b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-2kszh\" (UID: \"fcb6cd63-75f0-4efd-a878-fb7236f4109b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kszh"
Sep 15 20:09:28.777359 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:28.777224    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0c65a532-9608-4066-9b1a-156727664921-metrics-tls\") pod \"dns-default-qg2bs\" (UID: \"0c65a532-9608-4066-9b1a-156727664921\") " pod="openshift-dns/dns-default-qg2bs"
Sep 15 20:09:28.925790 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:28.925750    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-hdqjv\""
Sep 15 20:09:28.925988 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:28.925813    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jwzj9\""
Sep 15 20:09:28.925988 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:28.925766    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-g82mz\""
Sep 15 20:09:28.932536 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:28.932513    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qg2bs"
Sep 15 20:09:28.932536 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:28.932527    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:09:28.932689 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:28.932556    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kszh"
Sep 15 20:09:28.937691 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:28.937672    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ns6zw" event={"ID":"d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f","Type":"ContainerStarted","Data":"9935a7236e99b854053ca4cc13934c56c00884d283e198b6efa2be8ec97e7911"}
Sep 15 20:09:28.938031 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:28.937746    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ns6zw" event={"ID":"d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f","Type":"ContainerStarted","Data":"337d1b55cb2158c48529fc54c932b297f4fff9e7832be690e0c56dff7c13ba07"}
Sep 15 20:09:28.939103 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:28.939082    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-87qsf" event={"ID":"33eaf6a7-ab22-48a4-9502-3fc7943c9c27","Type":"ContainerStarted","Data":"9a1f3e828df21215904bc975a36a2277c3a8257f27c60b04a1815ae36e104b98"}
Sep 15 20:09:28.961793 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:28.961688    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-87qsf" podStartSLOduration=1.7287380730000002 podStartE2EDuration="2.961664921s" totalImagesPullingTime="1.232926848s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:09:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:09:28.960839373 +0000 UTC m=+283.449303851" watchObservedRunningTime="2026-09-15 20:09:28.961664921 +0000 UTC m=+283.450129399"
Sep 15 20:09:29.209468 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:29.209353    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qg2bs"]
Sep 15 20:09:29.220294 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:29.220271    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-557757c5cd-9l2fg"]
Sep 15 20:09:29.225700 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:29.225676    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-2kszh"]
Sep 15 20:09:29.245848 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:09:29.245819    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c65a532_9608_4066_9b1a_156727664921.slice/crio-6c3929e8960b2503b9a6f83886abae98cac555ecfd9c994e0ec28810e6eb9708 WatchSource:0}: Error finding container 6c3929e8960b2503b9a6f83886abae98cac555ecfd9c994e0ec28810e6eb9708: Status 404 returned error can't find the container with id 6c3929e8960b2503b9a6f83886abae98cac555ecfd9c994e0ec28810e6eb9708
Sep 15 20:09:29.246342 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:09:29.246315    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cb71e8f_b673_49d6_bce3_2b205b80b22f.slice/crio-653711d0d675a56777ef4af3146089300c6dbf3482204a193226f6f7ed850db7 WatchSource:0}: Error finding container 653711d0d675a56777ef4af3146089300c6dbf3482204a193226f6f7ed850db7: Status 404 returned error can't find the container with id 653711d0d675a56777ef4af3146089300c6dbf3482204a193226f6f7ed850db7
Sep 15 20:09:29.247222 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:09:29.247200    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcb6cd63_75f0_4efd_a878_fb7236f4109b.slice/crio-41fbc1816e8bd8813dafbaee6fdf0e7cbe17592214c89a26215390bd841aadf2 WatchSource:0}: Error finding container 41fbc1816e8bd8813dafbaee6fdf0e7cbe17592214c89a26215390bd841aadf2: Status 404 returned error can't find the container with id 41fbc1816e8bd8813dafbaee6fdf0e7cbe17592214c89a26215390bd841aadf2
Sep 15 20:09:29.944658 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:29.944619    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kszh" event={"ID":"fcb6cd63-75f0-4efd-a878-fb7236f4109b","Type":"ContainerStarted","Data":"41fbc1816e8bd8813dafbaee6fdf0e7cbe17592214c89a26215390bd841aadf2"}
Sep 15 20:09:29.946678 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:29.946635    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-557757c5cd-9l2fg" event={"ID":"9cb71e8f-b673-49d6-bce3-2b205b80b22f","Type":"ContainerStarted","Data":"03b7f5de8962492d4d8830f0ed29dedef67895179f350323e82afac5c279d174"}
Sep 15 20:09:29.946838 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:29.946681    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-557757c5cd-9l2fg" event={"ID":"9cb71e8f-b673-49d6-bce3-2b205b80b22f","Type":"ContainerStarted","Data":"653711d0d675a56777ef4af3146089300c6dbf3482204a193226f6f7ed850db7"}
Sep 15 20:09:29.946838 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:29.946752    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:09:29.948066 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:29.948035    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qg2bs" event={"ID":"0c65a532-9608-4066-9b1a-156727664921","Type":"ContainerStarted","Data":"6c3929e8960b2503b9a6f83886abae98cac555ecfd9c994e0ec28810e6eb9708"}
Sep 15 20:09:29.951042 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:29.951018    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ns6zw" event={"ID":"d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f","Type":"ContainerStarted","Data":"4e11395413aa046301066662ad76e60f1f94e79090b6771507756e64804eca66"}
Sep 15 20:09:29.951142 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:29.951075    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-87qsf"
Sep 15 20:09:29.956523 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:29.956504    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-87qsf"
Sep 15 20:09:29.970909 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:29.970834    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-557757c5cd-9l2fg" podStartSLOduration=283.970816991 podStartE2EDuration="4m43.970816991s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:04:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:09:29.969478003 +0000 UTC m=+284.457942479" watchObservedRunningTime="2026-09-15 20:09:29.970816991 +0000 UTC m=+284.459281469"
Sep 15 20:09:30.848075 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:30.848030    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:30.915323 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:30.915295    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-s9xjv"]
Sep 15 20:09:30.918469 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:30.918450    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-s9xjv"
Sep 15 20:09:30.921516 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:30.921470    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-m8lcs\""
Sep 15 20:09:30.921516 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:30.921485    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 15 20:09:30.921666 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:30.921490    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 15 20:09:30.921666 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:30.921470    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 15 20:09:30.927487 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:30.926618    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-s9xjv"]
Sep 15 20:09:30.942259 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:30.942200    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:31.063249 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:31.063218    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:31.093323 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:31.093297    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pkz7\" (UniqueName: \"kubernetes.io/projected/a58414ce-141f-46ee-b7de-a3735aae50f7-kube-api-access-2pkz7\") pod \"prometheus-operator-68bcd7f987-s9xjv\" (UID: \"a58414ce-141f-46ee-b7de-a3735aae50f7\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-s9xjv"
Sep 15 20:09:31.093464 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:31.093341    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a58414ce-141f-46ee-b7de-a3735aae50f7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-s9xjv\" (UID: \"a58414ce-141f-46ee-b7de-a3735aae50f7\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-s9xjv"
Sep 15 20:09:31.093529 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:31.093485    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a58414ce-141f-46ee-b7de-a3735aae50f7-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-s9xjv\" (UID: \"a58414ce-141f-46ee-b7de-a3735aae50f7\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-s9xjv"
Sep 15 20:09:31.093790 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:31.093754    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a58414ce-141f-46ee-b7de-a3735aae50f7-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-s9xjv\" (UID: \"a58414ce-141f-46ee-b7de-a3735aae50f7\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-s9xjv"
Sep 15 20:09:31.195192 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:31.195108    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a58414ce-141f-46ee-b7de-a3735aae50f7-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-s9xjv\" (UID: \"a58414ce-141f-46ee-b7de-a3735aae50f7\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-s9xjv"
Sep 15 20:09:31.195192 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:31.195165    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pkz7\" (UniqueName: \"kubernetes.io/projected/a58414ce-141f-46ee-b7de-a3735aae50f7-kube-api-access-2pkz7\") pod \"prometheus-operator-68bcd7f987-s9xjv\" (UID: \"a58414ce-141f-46ee-b7de-a3735aae50f7\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-s9xjv"
Sep 15 20:09:31.195407 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:31.195219    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a58414ce-141f-46ee-b7de-a3735aae50f7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-s9xjv\" (UID: \"a58414ce-141f-46ee-b7de-a3735aae50f7\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-s9xjv"
Sep 15 20:09:31.195407 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:31.195285    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a58414ce-141f-46ee-b7de-a3735aae50f7-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-s9xjv\" (UID: \"a58414ce-141f-46ee-b7de-a3735aae50f7\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-s9xjv"
Sep 15 20:09:31.196056 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:31.196028    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a58414ce-141f-46ee-b7de-a3735aae50f7-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-s9xjv\" (UID: \"a58414ce-141f-46ee-b7de-a3735aae50f7\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-s9xjv"
Sep 15 20:09:31.197836 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:31.197812    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a58414ce-141f-46ee-b7de-a3735aae50f7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-s9xjv\" (UID: \"a58414ce-141f-46ee-b7de-a3735aae50f7\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-s9xjv"
Sep 15 20:09:31.198109 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:31.198085    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a58414ce-141f-46ee-b7de-a3735aae50f7-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-s9xjv\" (UID: \"a58414ce-141f-46ee-b7de-a3735aae50f7\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-s9xjv"
Sep 15 20:09:31.203886 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:31.203840    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pkz7\" (UniqueName: \"kubernetes.io/projected/a58414ce-141f-46ee-b7de-a3735aae50f7-kube-api-access-2pkz7\") pod \"prometheus-operator-68bcd7f987-s9xjv\" (UID: \"a58414ce-141f-46ee-b7de-a3735aae50f7\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-s9xjv"
Sep 15 20:09:31.230660 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:31.230634    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-s9xjv"
Sep 15 20:09:31.797819 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:31.797788    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-s9xjv"]
Sep 15 20:09:31.799826 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:09:31.799801    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda58414ce_141f_46ee_b7de_a3735aae50f7.slice/crio-4f6d6b77bbd3982fb6970385ce93f3b9a4522103d272d04f5a4e89cacf1c3402 WatchSource:0}: Error finding container 4f6d6b77bbd3982fb6970385ce93f3b9a4522103d272d04f5a4e89cacf1c3402: Status 404 returned error can't find the container with id 4f6d6b77bbd3982fb6970385ce93f3b9a4522103d272d04f5a4e89cacf1c3402
Sep 15 20:09:31.960796 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:31.960765    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ns6zw" event={"ID":"d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f","Type":"ContainerStarted","Data":"b9fc1099f69007a5f2a6c9005e5dd7825d22665e738eefdc9a458ff569d1946e"}
Sep 15 20:09:31.961875 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:31.961846    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-s9xjv" event={"ID":"a58414ce-141f-46ee-b7de-a3735aae50f7","Type":"ContainerStarted","Data":"4f6d6b77bbd3982fb6970385ce93f3b9a4522103d272d04f5a4e89cacf1c3402"}
Sep 15 20:09:31.963154 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:31.963122    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kszh" event={"ID":"fcb6cd63-75f0-4efd-a878-fb7236f4109b","Type":"ContainerStarted","Data":"b947253d52fcbfe16be8d7f25d2c6d740493239f43ebbb73a50c6fd47ad3ab8a"}
Sep 15 20:09:31.964878 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:31.964856    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qg2bs" event={"ID":"0c65a532-9608-4066-9b1a-156727664921","Type":"ContainerStarted","Data":"63ae8f0cf0de3506341d80dc346cc43318c97013bc5b23e78193dc54fa1decab"}
Sep 15 20:09:31.964987 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:31.964896    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qg2bs" event={"ID":"0c65a532-9608-4066-9b1a-156727664921","Type":"ContainerStarted","Data":"2b9742a10f75a8b398b424a33f00b8b44c799a883141fffe7d9caa7dc23d503a"}
Sep 15 20:09:31.982549 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:31.982506    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ns6zw" podStartSLOduration=33.150793706 podStartE2EDuration="35.982494497s" totalImagesPullingTime="2.831700791s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:08:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:09:31.981863027 +0000 UTC m=+286.470327502" watchObservedRunningTime="2026-09-15 20:09:31.982494497 +0000 UTC m=+286.470958972"
Sep 15 20:09:31.998226 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:31.998185    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2kszh" podStartSLOduration=265.719398348 podStartE2EDuration="4m27.998174235s" totalImagesPullingTime="2.278775887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:05:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:09:31.997023698 +0000 UTC m=+286.485488174" watchObservedRunningTime="2026-09-15 20:09:31.998174235 +0000 UTC m=+286.486638734"
Sep 15 20:09:32.015424 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:32.015381    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-qg2bs" podStartSLOduration=251.732989257 podStartE2EDuration="4m14.015367681s" totalImagesPullingTime="2.282378424s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:05:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:09:32.014175152 +0000 UTC m=+286.502639639" watchObservedRunningTime="2026-09-15 20:09:32.015367681 +0000 UTC m=+286.503832156"
Sep 15 20:09:32.116399 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:32.116333    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:32.183892 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:32.183868    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:32.969395 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:32.969361    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-qg2bs"
Sep 15 20:09:33.488388 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:33.488318    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:33.974780 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:33.974746    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-s9xjv" event={"ID":"a58414ce-141f-46ee-b7de-a3735aae50f7","Type":"ContainerStarted","Data":"f69003fcbcb3a4d1f83e68392cb169c7d919771f5b9fbcf91f31c565c7fe579b"}
Sep 15 20:09:33.974942 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:33.974801    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-s9xjv" event={"ID":"a58414ce-141f-46ee-b7de-a3735aae50f7","Type":"ContainerStarted","Data":"be07cf7c0de07f71ef03baffda1d25593552a4e0edeafa802506102107d05e3c"}
Sep 15 20:09:33.992687 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:33.992643    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-s9xjv" podStartSLOduration=2.588366343 podStartE2EDuration="3.992628169s" totalImagesPullingTime="1.404261826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:09:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:09:33.991447111 +0000 UTC m=+288.479911569" watchObservedRunningTime="2026-09-15 20:09:33.992628169 +0000 UTC m=+288.481092645"
Sep 15 20:09:35.378216 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:35.378184    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:35.474906 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:35.474877    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:36.016140 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.016107    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:36.211803 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.211757    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:36.250103 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.250076    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-cb4km"]
Sep 15 20:09:36.254567 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.254551    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cb4km"
Sep 15 20:09:36.257304 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.257105    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 15 20:09:36.257304 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.257200    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 15 20:09:36.257304 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.257203    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-45rwz\""
Sep 15 20:09:36.266273 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.266220    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-cb4km"]
Sep 15 20:09:36.279086 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.279063    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-shd92"]
Sep 15 20:09:36.288055 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.288031    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-shd92"
Sep 15 20:09:36.290657 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.290639    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 15 20:09:36.290984 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.290966    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-pb2x8\""
Sep 15 20:09:36.291203 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.291057    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 15 20:09:36.291412 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.291384    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 15 20:09:36.296646 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.296625    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-tvkc6"]
Sep 15 20:09:36.300160 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.300140    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-shd92"]
Sep 15 20:09:36.300257 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.300219    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tvkc6"
Sep 15 20:09:36.303255 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.303235    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9jdsl\""
Sep 15 20:09:36.303398 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.303381    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 15 20:09:36.303482 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.303428    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 15 20:09:36.303709 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.303687    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 15 20:09:36.318700 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.318677    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:36.334628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.334606    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ffe68d8d-4681-4754-bf5c-a0ef28de8037-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-shd92\" (UID: \"ffe68d8d-4681-4754-bf5c-a0ef28de8037\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-shd92"
Sep 15 20:09:36.334717 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.334640    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/69d39979-106e-4316-91e4-6dfa7546a724-node-exporter-wtmp\") pod \"node-exporter-tvkc6\" (UID: \"69d39979-106e-4316-91e4-6dfa7546a724\") " pod="openshift-monitoring/node-exporter-tvkc6"
Sep 15 20:09:36.334717 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.334665    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/69d39979-106e-4316-91e4-6dfa7546a724-node-exporter-accelerators-collector-config\") pod \"node-exporter-tvkc6\" (UID: \"69d39979-106e-4316-91e4-6dfa7546a724\") " pod="openshift-monitoring/node-exporter-tvkc6"
Sep 15 20:09:36.334802 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.334714    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ffe68d8d-4681-4754-bf5c-a0ef28de8037-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-shd92\" (UID: \"ffe68d8d-4681-4754-bf5c-a0ef28de8037\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-shd92"
Sep 15 20:09:36.334802 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.334750    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f6a0afa-7e21-48f1-9173-6a0fbc184be7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-cb4km\" (UID: \"2f6a0afa-7e21-48f1-9173-6a0fbc184be7\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cb4km"
Sep 15 20:09:36.334886 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.334803    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69d39979-106e-4316-91e4-6dfa7546a724-metrics-client-ca\") pod \"node-exporter-tvkc6\" (UID: \"69d39979-106e-4316-91e4-6dfa7546a724\") " pod="openshift-monitoring/node-exporter-tvkc6"
Sep 15 20:09:36.334886 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.334832    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f6a0afa-7e21-48f1-9173-6a0fbc184be7-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-cb4km\" (UID: \"2f6a0afa-7e21-48f1-9173-6a0fbc184be7\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cb4km"
Sep 15 20:09:36.334886 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.334854    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/69d39979-106e-4316-91e4-6dfa7546a724-root\") pod \"node-exporter-tvkc6\" (UID: \"69d39979-106e-4316-91e4-6dfa7546a724\") " pod="openshift-monitoring/node-exporter-tvkc6"
Sep 15 20:09:36.334886 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.334879    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/69d39979-106e-4316-91e4-6dfa7546a724-node-exporter-textfile\") pod \"node-exporter-tvkc6\" (UID: \"69d39979-106e-4316-91e4-6dfa7546a724\") " pod="openshift-monitoring/node-exporter-tvkc6"
Sep 15 20:09:36.335074 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.334905    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/69d39979-106e-4316-91e4-6dfa7546a724-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tvkc6\" (UID: \"69d39979-106e-4316-91e4-6dfa7546a724\") " pod="openshift-monitoring/node-exporter-tvkc6"
Sep 15 20:09:36.335074 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.334952    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lct9x\" (UniqueName: \"kubernetes.io/projected/ffe68d8d-4681-4754-bf5c-a0ef28de8037-kube-api-access-lct9x\") pod \"kube-state-metrics-59fbdd77d-shd92\" (UID: \"ffe68d8d-4681-4754-bf5c-a0ef28de8037\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-shd92"
Sep 15 20:09:36.335074 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.334978    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ffe68d8d-4681-4754-bf5c-a0ef28de8037-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-shd92\" (UID: \"ffe68d8d-4681-4754-bf5c-a0ef28de8037\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-shd92"
Sep 15 20:09:36.335074 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.335005    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ffe68d8d-4681-4754-bf5c-a0ef28de8037-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-shd92\" (UID: \"ffe68d8d-4681-4754-bf5c-a0ef28de8037\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-shd92"
Sep 15 20:09:36.335074 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.335035    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx865\" (UniqueName: \"kubernetes.io/projected/2f6a0afa-7e21-48f1-9173-6a0fbc184be7-kube-api-access-hx865\") pod \"openshift-state-metrics-67dcf8c979-cb4km\" (UID: \"2f6a0afa-7e21-48f1-9173-6a0fbc184be7\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cb4km"
Sep 15 20:09:36.335314 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.335125    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sgmp\" (UniqueName: \"kubernetes.io/projected/69d39979-106e-4316-91e4-6dfa7546a724-kube-api-access-8sgmp\") pod \"node-exporter-tvkc6\" (UID: \"69d39979-106e-4316-91e4-6dfa7546a724\") " pod="openshift-monitoring/node-exporter-tvkc6"
Sep 15 20:09:36.335314 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.335160    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ffe68d8d-4681-4754-bf5c-a0ef28de8037-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-shd92\" (UID: \"ffe68d8d-4681-4754-bf5c-a0ef28de8037\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-shd92"
Sep 15 20:09:36.335314 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.335224    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/69d39979-106e-4316-91e4-6dfa7546a724-sys\") pod \"node-exporter-tvkc6\" (UID: \"69d39979-106e-4316-91e4-6dfa7546a724\") " pod="openshift-monitoring/node-exporter-tvkc6"
Sep 15 20:09:36.335314 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.335264    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/69d39979-106e-4316-91e4-6dfa7546a724-node-exporter-tls\") pod \"node-exporter-tvkc6\" (UID: \"69d39979-106e-4316-91e4-6dfa7546a724\") " pod="openshift-monitoring/node-exporter-tvkc6"
Sep 15 20:09:36.335314 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.335292    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2f6a0afa-7e21-48f1-9173-6a0fbc184be7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-cb4km\" (UID: \"2f6a0afa-7e21-48f1-9173-6a0fbc184be7\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cb4km"
Sep 15 20:09:36.436305 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.436280    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sgmp\" (UniqueName: \"kubernetes.io/projected/69d39979-106e-4316-91e4-6dfa7546a724-kube-api-access-8sgmp\") pod \"node-exporter-tvkc6\" (UID: \"69d39979-106e-4316-91e4-6dfa7546a724\") " pod="openshift-monitoring/node-exporter-tvkc6"
Sep 15 20:09:36.436697 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.436316    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ffe68d8d-4681-4754-bf5c-a0ef28de8037-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-shd92\" (UID: \"ffe68d8d-4681-4754-bf5c-a0ef28de8037\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-shd92"
Sep 15 20:09:36.436697 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.436362    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/69d39979-106e-4316-91e4-6dfa7546a724-sys\") pod \"node-exporter-tvkc6\" (UID: \"69d39979-106e-4316-91e4-6dfa7546a724\") " pod="openshift-monitoring/node-exporter-tvkc6"
Sep 15 20:09:36.436697 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.436406    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/69d39979-106e-4316-91e4-6dfa7546a724-node-exporter-tls\") pod \"node-exporter-tvkc6\" (UID: \"69d39979-106e-4316-91e4-6dfa7546a724\") " pod="openshift-monitoring/node-exporter-tvkc6"
Sep 15 20:09:36.436697 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.436437    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2f6a0afa-7e21-48f1-9173-6a0fbc184be7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-cb4km\" (UID: \"2f6a0afa-7e21-48f1-9173-6a0fbc184be7\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cb4km"
Sep 15 20:09:36.436697 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.436466    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ffe68d8d-4681-4754-bf5c-a0ef28de8037-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-shd92\" (UID: \"ffe68d8d-4681-4754-bf5c-a0ef28de8037\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-shd92"
Sep 15 20:09:36.436697 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.436489    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/69d39979-106e-4316-91e4-6dfa7546a724-node-exporter-wtmp\") pod \"node-exporter-tvkc6\" (UID: \"69d39979-106e-4316-91e4-6dfa7546a724\") " pod="openshift-monitoring/node-exporter-tvkc6"
Sep 15 20:09:36.436697 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.436514    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/69d39979-106e-4316-91e4-6dfa7546a724-node-exporter-accelerators-collector-config\") pod \"node-exporter-tvkc6\" (UID: \"69d39979-106e-4316-91e4-6dfa7546a724\") " pod="openshift-monitoring/node-exporter-tvkc6"
Sep 15 20:09:36.436697 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.436539    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ffe68d8d-4681-4754-bf5c-a0ef28de8037-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-shd92\" (UID: \"ffe68d8d-4681-4754-bf5c-a0ef28de8037\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-shd92"
Sep 15 20:09:36.436697 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.436572    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f6a0afa-7e21-48f1-9173-6a0fbc184be7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-cb4km\" (UID: \"2f6a0afa-7e21-48f1-9173-6a0fbc184be7\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cb4km"
Sep 15 20:09:36.436697 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.436616    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69d39979-106e-4316-91e4-6dfa7546a724-metrics-client-ca\") pod \"node-exporter-tvkc6\" (UID: \"69d39979-106e-4316-91e4-6dfa7546a724\") " pod="openshift-monitoring/node-exporter-tvkc6"
Sep 15 20:09:36.436697 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.436642    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f6a0afa-7e21-48f1-9173-6a0fbc184be7-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-cb4km\" (UID: \"2f6a0afa-7e21-48f1-9173-6a0fbc184be7\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cb4km"
Sep 15 20:09:36.436697 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.436664    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/69d39979-106e-4316-91e4-6dfa7546a724-root\") pod \"node-exporter-tvkc6\" (UID: \"69d39979-106e-4316-91e4-6dfa7546a724\") " pod="openshift-monitoring/node-exporter-tvkc6"
Sep 15 20:09:36.436697 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.436687    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/69d39979-106e-4316-91e4-6dfa7546a724-node-exporter-textfile\") pod \"node-exporter-tvkc6\" (UID: \"69d39979-106e-4316-91e4-6dfa7546a724\") " pod="openshift-monitoring/node-exporter-tvkc6"
Sep 15 20:09:36.437411 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.436713    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/69d39979-106e-4316-91e4-6dfa7546a724-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tvkc6\" (UID: \"69d39979-106e-4316-91e4-6dfa7546a724\") " pod="openshift-monitoring/node-exporter-tvkc6"
Sep 15 20:09:36.437411 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.436737    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lct9x\" (UniqueName: \"kubernetes.io/projected/ffe68d8d-4681-4754-bf5c-a0ef28de8037-kube-api-access-lct9x\") pod \"kube-state-metrics-59fbdd77d-shd92\" (UID: \"ffe68d8d-4681-4754-bf5c-a0ef28de8037\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-shd92"
Sep 15 20:09:36.437411 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.436764    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ffe68d8d-4681-4754-bf5c-a0ef28de8037-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-shd92\" (UID: \"ffe68d8d-4681-4754-bf5c-a0ef28de8037\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-shd92"
Sep 15 20:09:36.437411 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.436791    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ffe68d8d-4681-4754-bf5c-a0ef28de8037-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-shd92\" (UID: \"ffe68d8d-4681-4754-bf5c-a0ef28de8037\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-shd92"
Sep 15 20:09:36.437411 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.436824    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx865\" (UniqueName: \"kubernetes.io/projected/2f6a0afa-7e21-48f1-9173-6a0fbc184be7-kube-api-access-hx865\") pod \"openshift-state-metrics-67dcf8c979-cb4km\" (UID: \"2f6a0afa-7e21-48f1-9173-6a0fbc184be7\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cb4km"
Sep 15 20:09:36.437411 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.436936    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ffe68d8d-4681-4754-bf5c-a0ef28de8037-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-shd92\" (UID: \"ffe68d8d-4681-4754-bf5c-a0ef28de8037\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-shd92"
Sep 15 20:09:36.437411 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.437003    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/69d39979-106e-4316-91e4-6dfa7546a724-sys\") pod \"node-exporter-tvkc6\" (UID: \"69d39979-106e-4316-91e4-6dfa7546a724\") " pod="openshift-monitoring/node-exporter-tvkc6"
Sep 15 20:09:36.437411 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:09:36.437085    2576 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 15 20:09:36.437411 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:09:36.437136    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69d39979-106e-4316-91e4-6dfa7546a724-node-exporter-tls podName:69d39979-106e-4316-91e4-6dfa7546a724 nodeName:}" failed. No retries permitted until 2026-09-15 20:09:36.937118268 +0000 UTC m=+291.425582727 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/69d39979-106e-4316-91e4-6dfa7546a724-node-exporter-tls") pod "node-exporter-tvkc6" (UID: "69d39979-106e-4316-91e4-6dfa7546a724") : secret "node-exporter-tls" not found
Sep 15 20:09:36.437411 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.437216    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/69d39979-106e-4316-91e4-6dfa7546a724-node-exporter-wtmp\") pod \"node-exporter-tvkc6\" (UID: \"69d39979-106e-4316-91e4-6dfa7546a724\") " pod="openshift-monitoring/node-exporter-tvkc6"
Sep 15 20:09:36.439293 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.437648    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/69d39979-106e-4316-91e4-6dfa7546a724-root\") pod \"node-exporter-tvkc6\" (UID: \"69d39979-106e-4316-91e4-6dfa7546a724\") " pod="openshift-monitoring/node-exporter-tvkc6"
Sep 15 20:09:36.439293 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.437762    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/69d39979-106e-4316-91e4-6dfa7546a724-node-exporter-accelerators-collector-config\") pod \"node-exporter-tvkc6\" (UID: \"69d39979-106e-4316-91e4-6dfa7546a724\") " pod="openshift-monitoring/node-exporter-tvkc6"
Sep 15 20:09:36.439293 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.438117    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/69d39979-106e-4316-91e4-6dfa7546a724-node-exporter-textfile\") pod \"node-exporter-tvkc6\" (UID: \"69d39979-106e-4316-91e4-6dfa7546a724\") " pod="openshift-monitoring/node-exporter-tvkc6"
Sep 15 20:09:36.439293 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.438131    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69d39979-106e-4316-91e4-6dfa7546a724-metrics-client-ca\") pod \"node-exporter-tvkc6\" (UID: \"69d39979-106e-4316-91e4-6dfa7546a724\") " pod="openshift-monitoring/node-exporter-tvkc6"
Sep 15 20:09:36.439293 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:09:36.438384    2576 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 15 20:09:36.439293 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:09:36.438473    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffe68d8d-4681-4754-bf5c-a0ef28de8037-kube-state-metrics-tls podName:ffe68d8d-4681-4754-bf5c-a0ef28de8037 nodeName:}" failed. No retries permitted until 2026-09-15 20:09:36.938456614 +0000 UTC m=+291.426921068 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/ffe68d8d-4681-4754-bf5c-a0ef28de8037-kube-state-metrics-tls") pod "kube-state-metrics-59fbdd77d-shd92" (UID: "ffe68d8d-4681-4754-bf5c-a0ef28de8037") : secret "kube-state-metrics-tls" not found
Sep 15 20:09:36.439293 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:09:36.438497    2576 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 15 20:09:36.439293 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:09:36.438557    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f6a0afa-7e21-48f1-9173-6a0fbc184be7-openshift-state-metrics-tls podName:2f6a0afa-7e21-48f1-9173-6a0fbc184be7 nodeName:}" failed. No retries permitted until 2026-09-15 20:09:36.938540844 +0000 UTC m=+291.427005304 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/2f6a0afa-7e21-48f1-9173-6a0fbc184be7-openshift-state-metrics-tls") pod "openshift-state-metrics-67dcf8c979-cb4km" (UID: "2f6a0afa-7e21-48f1-9173-6a0fbc184be7") : secret "openshift-state-metrics-tls" not found
Sep 15 20:09:36.439293 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.438652    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ffe68d8d-4681-4754-bf5c-a0ef28de8037-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-shd92\" (UID: \"ffe68d8d-4681-4754-bf5c-a0ef28de8037\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-shd92"
Sep 15 20:09:36.439293 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.438811    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ffe68d8d-4681-4754-bf5c-a0ef28de8037-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-shd92\" (UID: \"ffe68d8d-4681-4754-bf5c-a0ef28de8037\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-shd92"
Sep 15 20:09:36.439797 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.439317    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ffe68d8d-4681-4754-bf5c-a0ef28de8037-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-shd92\" (UID: \"ffe68d8d-4681-4754-bf5c-a0ef28de8037\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-shd92"
Sep 15 20:09:36.439797 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.439706    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f6a0afa-7e21-48f1-9173-6a0fbc184be7-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-cb4km\" (UID: \"2f6a0afa-7e21-48f1-9173-6a0fbc184be7\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cb4km"
Sep 15 20:09:36.440905 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.440851    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/69d39979-106e-4316-91e4-6dfa7546a724-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tvkc6\" (UID: \"69d39979-106e-4316-91e4-6dfa7546a724\") " pod="openshift-monitoring/node-exporter-tvkc6"
Sep 15 20:09:36.440905 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.440870    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:36.441909 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.441884    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2f6a0afa-7e21-48f1-9173-6a0fbc184be7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-cb4km\" (UID: \"2f6a0afa-7e21-48f1-9173-6a0fbc184be7\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cb4km"
Sep 15 20:09:36.445480 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.445440    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sgmp\" (UniqueName: \"kubernetes.io/projected/69d39979-106e-4316-91e4-6dfa7546a724-kube-api-access-8sgmp\") pod \"node-exporter-tvkc6\" (UID: \"69d39979-106e-4316-91e4-6dfa7546a724\") " pod="openshift-monitoring/node-exporter-tvkc6"
Sep 15 20:09:36.446539 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.446519    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx865\" (UniqueName: \"kubernetes.io/projected/2f6a0afa-7e21-48f1-9173-6a0fbc184be7-kube-api-access-hx865\") pod \"openshift-state-metrics-67dcf8c979-cb4km\" (UID: \"2f6a0afa-7e21-48f1-9173-6a0fbc184be7\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cb4km"
Sep 15 20:09:36.451260 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.451235    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lct9x\" (UniqueName: \"kubernetes.io/projected/ffe68d8d-4681-4754-bf5c-a0ef28de8037-kube-api-access-lct9x\") pod \"kube-state-metrics-59fbdd77d-shd92\" (UID: \"ffe68d8d-4681-4754-bf5c-a0ef28de8037\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-shd92"
Sep 15 20:09:36.556589 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.556495    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:36.675223 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.675190    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:36.775425 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.775395    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:36.879899 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.879804    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:36.942448 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.942419    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/69d39979-106e-4316-91e4-6dfa7546a724-node-exporter-tls\") pod \"node-exporter-tvkc6\" (UID: \"69d39979-106e-4316-91e4-6dfa7546a724\") " pod="openshift-monitoring/node-exporter-tvkc6"
Sep 15 20:09:36.942576 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.942466    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ffe68d8d-4681-4754-bf5c-a0ef28de8037-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-shd92\" (UID: \"ffe68d8d-4681-4754-bf5c-a0ef28de8037\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-shd92"
Sep 15 20:09:36.942576 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.942497    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f6a0afa-7e21-48f1-9173-6a0fbc184be7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-cb4km\" (UID: \"2f6a0afa-7e21-48f1-9173-6a0fbc184be7\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cb4km"
Sep 15 20:09:36.944861 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.944836    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/69d39979-106e-4316-91e4-6dfa7546a724-node-exporter-tls\") pod \"node-exporter-tvkc6\" (UID: \"69d39979-106e-4316-91e4-6dfa7546a724\") " pod="openshift-monitoring/node-exporter-tvkc6"
Sep 15 20:09:36.944987 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.944894    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f6a0afa-7e21-48f1-9173-6a0fbc184be7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-cb4km\" (UID: \"2f6a0afa-7e21-48f1-9173-6a0fbc184be7\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cb4km"
Sep 15 20:09:36.945033 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:36.944979    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ffe68d8d-4681-4754-bf5c-a0ef28de8037-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-shd92\" (UID: \"ffe68d8d-4681-4754-bf5c-a0ef28de8037\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-shd92"
Sep 15 20:09:37.034394 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:37.034366    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:37.138463 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:09:37.138403    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-59mfs" podUID="3b8d9342-6574-4d0d-a326-0299eb3b4c5d"
Sep 15 20:09:37.164774 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:37.164755    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cb4km"
Sep 15 20:09:37.198014 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:37.197987    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-shd92"
Sep 15 20:09:37.211343 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:37.211187    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tvkc6"
Sep 15 20:09:37.266692 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:37.266535    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:37.363751 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:37.363720    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:37.443276 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:37.443203    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-cb4km"]
Sep 15 20:09:37.448351 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:09:37.448318    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f6a0afa_7e21_48f1_9173_6a0fbc184be7.slice/crio-bca123407f4da9b8964ed0a0dc6d5fdaac70c942e8b25c391bf4e1473fc01ce4 WatchSource:0}: Error finding container bca123407f4da9b8964ed0a0dc6d5fdaac70c942e8b25c391bf4e1473fc01ce4: Status 404 returned error can't find the container with id bca123407f4da9b8964ed0a0dc6d5fdaac70c942e8b25c391bf4e1473fc01ce4
Sep 15 20:09:37.482685 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:37.482661    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:37.496115 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:37.496092    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-shd92"]
Sep 15 20:09:37.508934 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:09:37.508874    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffe68d8d_4681_4754_bf5c_a0ef28de8037.slice/crio-ec7bde9a98e19f19889b043d1eb52f8be3967911eda1c13967ab5c4d1c63bc7b WatchSource:0}: Error finding container ec7bde9a98e19f19889b043d1eb52f8be3967911eda1c13967ab5c4d1c63bc7b: Status 404 returned error can't find the container with id ec7bde9a98e19f19889b043d1eb52f8be3967911eda1c13967ab5c4d1c63bc7b
Sep 15 20:09:37.707156 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:37.707130    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:37.771408 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:37.771387    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:37.991005 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:37.990900    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-shd92" event={"ID":"ffe68d8d-4681-4754-bf5c-a0ef28de8037","Type":"ContainerStarted","Data":"ec7bde9a98e19f19889b043d1eb52f8be3967911eda1c13967ab5c4d1c63bc7b"}
Sep 15 20:09:37.992272 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:37.992247    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tvkc6" event={"ID":"69d39979-106e-4316-91e4-6dfa7546a724","Type":"ContainerStarted","Data":"d1976f6995aec08cde804e9f7aae5cdc80d74bcc6a070bf409b9aac54c55dacd"}
Sep 15 20:09:37.994026 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:37.994006    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cb4km" event={"ID":"2f6a0afa-7e21-48f1-9173-6a0fbc184be7","Type":"ContainerStarted","Data":"f97ac96b338f5ee8d3bf63f27caa4eab3aa4227b711cd0b26797055044fbb050"}
Sep 15 20:09:37.994110 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:37.994049    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cb4km" event={"ID":"2f6a0afa-7e21-48f1-9173-6a0fbc184be7","Type":"ContainerStarted","Data":"d689b7adc94bde7fd85dfbae05bafc5edfd1dd5e76161f454ae2f03c3f516250"}
Sep 15 20:09:37.994110 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:37.994067    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cb4km" event={"ID":"2f6a0afa-7e21-48f1-9173-6a0fbc184be7","Type":"ContainerStarted","Data":"bca123407f4da9b8964ed0a0dc6d5fdaac70c942e8b25c391bf4e1473fc01ce4"}
Sep 15 20:09:39.243844 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:39.243816    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:39.331412 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:39.331379    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:39.713516 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:39.713488    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:39.917245 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:39.917212    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:40.005309 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:40.005227    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-shd92" event={"ID":"ffe68d8d-4681-4754-bf5c-a0ef28de8037","Type":"ContainerStarted","Data":"7ae4017f4947df7d9dc73a044e53f719a337ab58a4ccf1c284df60754608e395"}
Sep 15 20:09:40.005469 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:40.005314    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-shd92" event={"ID":"ffe68d8d-4681-4754-bf5c-a0ef28de8037","Type":"ContainerStarted","Data":"1dcbad41b32776f3c57c961bb528259cd998ec3365c85ca9909593f82bcaf631"}
Sep 15 20:09:40.005469 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:40.005345    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-shd92" event={"ID":"ffe68d8d-4681-4754-bf5c-a0ef28de8037","Type":"ContainerStarted","Data":"157c2ae10c93d6c91a7ba12453b6ac2007d495eea1e05f4945b9cbc65fc43beb"}
Sep 15 20:09:40.006817 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:40.006781    2576 generic.go:356] "Generic (PLEG): container finished" podID="69d39979-106e-4316-91e4-6dfa7546a724" containerID="6ef3137de5a05aa70568f5ca0f4be84d37c54c694dc7d637f9197ac006e51fd4" exitCode=0
Sep 15 20:09:40.006956 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:40.006852    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tvkc6" event={"ID":"69d39979-106e-4316-91e4-6dfa7546a724","Type":"ContainerDied","Data":"6ef3137de5a05aa70568f5ca0f4be84d37c54c694dc7d637f9197ac006e51fd4"}
Sep 15 20:09:40.009084 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:40.009060    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cb4km" event={"ID":"2f6a0afa-7e21-48f1-9173-6a0fbc184be7","Type":"ContainerStarted","Data":"a12d81c8483a74feccb7fc2824c85c09f39fecd4c7c16fb7c9fe4d7bb2675237"}
Sep 15 20:09:40.027531 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:40.027491    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-shd92" podStartSLOduration=2.350559569 podStartE2EDuration="4.027477964s" totalImagesPullingTime="1.676918395s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:09:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:09:40.026461148 +0000 UTC m=+294.514925622" watchObservedRunningTime="2026-09-15 20:09:40.027477964 +0000 UTC m=+294.515942434"
Sep 15 20:09:40.045758 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:40.045708    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cb4km" podStartSLOduration=2.186943574 podStartE2EDuration="4.045691122s" totalImagesPullingTime="1.858747548s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:09:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:09:40.045158901 +0000 UTC m=+294.533623381" watchObservedRunningTime="2026-09-15 20:09:40.045691122 +0000 UTC m=+294.534155599"
Sep 15 20:09:41.016237 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:41.016202    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tvkc6" event={"ID":"69d39979-106e-4316-91e4-6dfa7546a724","Type":"ContainerStarted","Data":"34099e32ad86e23268f756839ea2d10e78254ec65fca06d176bbe7ae40157d7a"}
Sep 15 20:09:41.016691 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:41.016265    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tvkc6" event={"ID":"69d39979-106e-4316-91e4-6dfa7546a724","Type":"ContainerStarted","Data":"2bfa04135491caf0a05fdb54544e3f623234bf2ab8c49bd8c0e992b8a8069111"}
Sep 15 20:09:41.094559 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:41.094513    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-tvkc6" podStartSLOduration=3.026792405 podStartE2EDuration="5.094497445s" totalImagesPullingTime="1.95478821s" totalInitContainerRuntime="112.91683ms" isStatefulPod=true podCreationTimestamp="2026-09-15 20:09:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:09:41.09384644 +0000 UTC m=+295.582310915" watchObservedRunningTime="2026-09-15 20:09:41.094497445 +0000 UTC m=+295.582961920"
Sep 15 20:09:41.106229 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:41.106201    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:41.201176 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:41.201143    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:41.299504 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:41.299430    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:41.398539 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:41.398513    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:42.481876 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.481842    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:42.486561 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.486535    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 20:09:42.490821 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.490794    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.493167 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.493148    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 15 20:09:42.493167 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.493173    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 15 20:09:42.493478 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.493464    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 15 20:09:42.493747 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.493725    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 15 20:09:42.493972 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.493947    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 15 20:09:42.495081 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.494261    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 15 20:09:42.495081 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.494313    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 15 20:09:42.495081 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.494420    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6h4m3k1d24rt4\""
Sep 15 20:09:42.495081 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.494455    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 15 20:09:42.495081 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.494879    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 15 20:09:42.496949 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.495789    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 15 20:09:42.496949 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.496870    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 15 20:09:42.497779 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.497675    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 15 20:09:42.498072 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.498035    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 15 20:09:42.502132 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.500563    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-s6d2x\""
Sep 15 20:09:42.502132 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.500632    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 15 20:09:42.503760 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.503396    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 15 20:09:42.512531 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.512510    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 20:09:42.599813 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.599773    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.599996 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.599825    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.599996 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.599881    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg8gr\" (UniqueName: \"kubernetes.io/projected/21554748-1332-4b7e-a9ff-df0b33c48b19-kube-api-access-xg8gr\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.599996 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.599945    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.599996 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.599980    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.600146 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.600007    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.600146 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.600043    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.600146 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.600060    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.600146 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.600088    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/21554748-1332-4b7e-a9ff-df0b33c48b19-config-out\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.600146 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.600140    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-web-config\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.600309 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.600177    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.600309 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.600211    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.600309 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.600237    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/21554748-1332-4b7e-a9ff-df0b33c48b19-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.600309 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.600265    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/21554748-1332-4b7e-a9ff-df0b33c48b19-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.600431 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.600316    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.600431 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.600356    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-config\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.600431 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.600380    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.600431 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.600406    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.600568 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.600470    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.600568 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.600520    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.701459 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.701422    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.701459 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.701456    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg8gr\" (UniqueName: \"kubernetes.io/projected/21554748-1332-4b7e-a9ff-df0b33c48b19-kube-api-access-xg8gr\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.701675 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.701478    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.701675 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.701611    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.701675 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.701644    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.701804 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.701685    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.701804 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.701711    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.701804 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.701743    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/21554748-1332-4b7e-a9ff-df0b33c48b19-config-out\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.702426 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.702398    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-web-config\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.702535 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.702439    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.702535 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.702470    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.702535 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.702496    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/21554748-1332-4b7e-a9ff-df0b33c48b19-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.702535 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.702528    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/21554748-1332-4b7e-a9ff-df0b33c48b19-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.702739 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.702563    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.702739 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.702575    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.702739 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.702592    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-config\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.702739 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.702620    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.702739 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.702653    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.702739 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.702701    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.703067 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.702764    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.703067 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.702812    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.703175 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.703074    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.704764 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.704050    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.704764 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.704091    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/21554748-1332-4b7e-a9ff-df0b33c48b19-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.704764 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.704637    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.704764 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.704731    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.707744 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.705566    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/21554748-1332-4b7e-a9ff-df0b33c48b19-config-out\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.707744 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.705777    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.707744 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.706137    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.707744 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.706616    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.708962 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.708522    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-config\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.708962 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.708565    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.708962 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.708577    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.708962 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.708824    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.708962 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.708899    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.709223 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.708976    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.709223 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.709095    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.709334 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.709258    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-web-config\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.709589 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.709546    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/21554748-1332-4b7e-a9ff-df0b33c48b19-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.710746 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.710727    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg8gr\" (UniqueName: \"kubernetes.io/projected/21554748-1332-4b7e-a9ff-df0b33c48b19-kube-api-access-xg8gr\") pod \"prometheus-k8s-0\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.807047 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.806965    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:42.977193 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:42.977147    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qg2bs"
Sep 15 20:09:43.098644 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:43.098562    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 20:09:43.101364 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:09:43.101339    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21554748_1332_4b7e_a9ff_df0b33c48b19.slice/crio-41598c076e03653b2133a383f3e5f72e74ca07c4e5a2f6cdec7240405ca65877 WatchSource:0}: Error finding container 41598c076e03653b2133a383f3e5f72e74ca07c4e5a2f6cdec7240405ca65877: Status 404 returned error can't find the container with id 41598c076e03653b2133a383f3e5f72e74ca07c4e5a2f6cdec7240405ca65877
Sep 15 20:09:43.725489 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:43.725458    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:44.030138 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:44.030065    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"21554748-1332-4b7e-a9ff-df0b33c48b19","Type":"ContainerStarted","Data":"41598c076e03653b2133a383f3e5f72e74ca07c4e5a2f6cdec7240405ca65877"}
Sep 15 20:09:44.338158 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:44.338069    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:44.438887 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:44.438818    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:45.034994 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:45.034964    2576 generic.go:356] "Generic (PLEG): container finished" podID="21554748-1332-4b7e-a9ff-df0b33c48b19" containerID="7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992" exitCode=0
Sep 15 20:09:45.035350 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:45.035049    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"21554748-1332-4b7e-a9ff-df0b33c48b19","Type":"ContainerDied","Data":"7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992"}
Sep 15 20:09:45.940048 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:45.940021    2576 kubelet.go:1648] "Image garbage collection succeeded"
Sep 15 20:09:48.937839 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:48.937795    2576 patch_prober.go:28] interesting pod/image-registry-557757c5cd-9l2fg container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 15 20:09:48.940644 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:48.937868    2576 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-557757c5cd-9l2fg" podUID="9cb71e8f-b673-49d6-bce3-2b205b80b22f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 20:09:49.269171 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:49.268773    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:49.393969 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:49.393947    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 20:09:49.751254 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:49.751223    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:49.853465 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:49.853435    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:50.058628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:50.058544    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"21554748-1332-4b7e-a9ff-df0b33c48b19","Type":"ContainerStarted","Data":"9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d"}
Sep 15 20:09:50.058628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:50.058620    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"21554748-1332-4b7e-a9ff-df0b33c48b19","Type":"ContainerStarted","Data":"7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1"}
Sep 15 20:09:50.137160 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:50.137128    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:09:50.959553 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:50.959530    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:09:51.215297 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:51.215211    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:51.316437 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:51.316194    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:52.069101 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:52.069076    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"21554748-1332-4b7e-a9ff-df0b33c48b19","Type":"ContainerStarted","Data":"1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7"}
Sep 15 20:09:52.069229 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:52.069137    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"21554748-1332-4b7e-a9ff-df0b33c48b19","Type":"ContainerStarted","Data":"5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20"}
Sep 15 20:09:52.069229 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:52.069170    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"21554748-1332-4b7e-a9ff-df0b33c48b19","Type":"ContainerStarted","Data":"e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7"}
Sep 15 20:09:52.069229 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:52.069194    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"21554748-1332-4b7e-a9ff-df0b33c48b19","Type":"ContainerStarted","Data":"1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c"}
Sep 15 20:09:52.099660 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:52.099604    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.489216206 podStartE2EDuration="10.099585147s" totalImagesPullingTime="7.59427274s" totalInitContainerRuntime="16.096201ms" isStatefulPod=false podCreationTimestamp="2026-09-15 20:09:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:09:52.098358545 +0000 UTC m=+306.586823020" watchObservedRunningTime="2026-09-15 20:09:52.099585147 +0000 UTC m=+306.588049624"
Sep 15 20:09:52.127347 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:52.127314    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:52.193316 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:52.193292    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:52.807940 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:52.807889    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:09:53.961478 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:53.961442    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-557757c5cd-9l2fg"]
Sep 15 20:09:54.066795 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:54.066761    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:54.166646 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:54.166615    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:54.470504 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:54.470473    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:54.542393 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:54.542355    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:54.880214 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:54.880128    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:54.945264 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:54.945230    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:55.132268 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:55.132177    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:55.233622 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:55.233589    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:55.466450 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:55.466425    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:09:55.526439 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:09:55.526413    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:10:02.274204 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:02.274173    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:10:02.330255 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:02.330224    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:10:03.146417 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:03.146380    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:10:03.243965 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:03.243915    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:10:03.304214 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:03.304181    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:10:03.372352 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:03.372324    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:10:05.554507 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:05.554475    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:10:06.078337 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:06.078298    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:10:06.125065 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:06.125034    2576 generic.go:356] "Generic (PLEG): container finished" podID="93c1a30d-050c-491d-adbc-b5e876968efd" containerID="a2543a5032ebf170a0b08f991ec10d58b8a80dee0f190954727b85f08eca0b4c" exitCode=0
Sep 15 20:10:06.125214 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:06.125098    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-9cxxk" event={"ID":"93c1a30d-050c-491d-adbc-b5e876968efd","Type":"ContainerDied","Data":"a2543a5032ebf170a0b08f991ec10d58b8a80dee0f190954727b85f08eca0b4c"}
Sep 15 20:10:06.125465 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:06.125451    2576 scope.go:117] "RemoveContainer" containerID="a2543a5032ebf170a0b08f991ec10d58b8a80dee0f190954727b85f08eca0b4c"
Sep 15 20:10:07.130639 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:07.130608    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-9cxxk" event={"ID":"93c1a30d-050c-491d-adbc-b5e876968efd","Type":"ContainerStarted","Data":"00bc24d25b2db67759714626c2d914f395a82fb2e4389bb6b4750343fe83efca"}
Sep 15 20:10:15.208038 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:15.208002    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:10:15.266881 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:15.266854    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:10:18.986329 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:18.986284    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-557757c5cd-9l2fg" podUID="9cb71e8f-b673-49d6-bce3-2b205b80b22f" containerName="registry" containerID="cri-o://03b7f5de8962492d4d8830f0ed29dedef67895179f350323e82afac5c279d174" gracePeriod=30
Sep 15 20:10:19.182037 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:19.182007    2576 generic.go:356] "Generic (PLEG): container finished" podID="9cb71e8f-b673-49d6-bce3-2b205b80b22f" containerID="03b7f5de8962492d4d8830f0ed29dedef67895179f350323e82afac5c279d174" exitCode=0
Sep 15 20:10:19.182191 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:19.182108    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-557757c5cd-9l2fg" event={"ID":"9cb71e8f-b673-49d6-bce3-2b205b80b22f","Type":"ContainerDied","Data":"03b7f5de8962492d4d8830f0ed29dedef67895179f350323e82afac5c279d174"}
Sep 15 20:10:19.249470 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:19.249416    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:10:19.323996 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:19.323963    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-tls\") pod \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") "
Sep 15 20:10:19.324148 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:19.324029    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-bound-sa-token\") pod \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") "
Sep 15 20:10:19.324208 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:19.324187    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9cb71e8f-b673-49d6-bce3-2b205b80b22f-ca-trust-extracted\") pod \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") "
Sep 15 20:10:19.324262 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:19.324234    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9cb71e8f-b673-49d6-bce3-2b205b80b22f-trusted-ca\") pod \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") "
Sep 15 20:10:19.324322 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:19.324281    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-certificates\") pod \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") "
Sep 15 20:10:19.324322 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:19.324314    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lw4gn\" (UniqueName: \"kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-kube-api-access-lw4gn\") pod \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") "
Sep 15 20:10:19.324417 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:19.324392    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9cb71e8f-b673-49d6-bce3-2b205b80b22f-installation-pull-secrets\") pod \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") "
Sep 15 20:10:19.324466 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:19.324432    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9cb71e8f-b673-49d6-bce3-2b205b80b22f-image-registry-private-configuration\") pod \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\" (UID: \"9cb71e8f-b673-49d6-bce3-2b205b80b22f\") "
Sep 15 20:10:19.324770 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:19.324664    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "9cb71e8f-b673-49d6-bce3-2b205b80b22f" (UID: "9cb71e8f-b673-49d6-bce3-2b205b80b22f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:10:19.324770 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:19.324690    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cb71e8f-b673-49d6-bce3-2b205b80b22f-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "9cb71e8f-b673-49d6-bce3-2b205b80b22f" (UID: "9cb71e8f-b673-49d6-bce3-2b205b80b22f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:10:19.326672 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:19.326625    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-kube-api-access-lw4gn" (OuterVolumeSpecName: "kube-api-access-lw4gn") pod "9cb71e8f-b673-49d6-bce3-2b205b80b22f" (UID: "9cb71e8f-b673-49d6-bce3-2b205b80b22f"). InnerVolumeSpecName "kube-api-access-lw4gn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:10:19.326972 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:19.326887    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9cb71e8f-b673-49d6-bce3-2b205b80b22f" (UID: "9cb71e8f-b673-49d6-bce3-2b205b80b22f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:10:19.327055 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:19.326984    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9cb71e8f-b673-49d6-bce3-2b205b80b22f-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "9cb71e8f-b673-49d6-bce3-2b205b80b22f" (UID: "9cb71e8f-b673-49d6-bce3-2b205b80b22f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:10:19.327307 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:19.327281    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9cb71e8f-b673-49d6-bce3-2b205b80b22f-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "9cb71e8f-b673-49d6-bce3-2b205b80b22f" (UID: "9cb71e8f-b673-49d6-bce3-2b205b80b22f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:10:19.327385 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:19.327362    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "9cb71e8f-b673-49d6-bce3-2b205b80b22f" (UID: "9cb71e8f-b673-49d6-bce3-2b205b80b22f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:10:19.333489 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:19.333463    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cb71e8f-b673-49d6-bce3-2b205b80b22f-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "9cb71e8f-b673-49d6-bce3-2b205b80b22f" (UID: "9cb71e8f-b673-49d6-bce3-2b205b80b22f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:10:19.425370 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:19.425333    2576 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9cb71e8f-b673-49d6-bce3-2b205b80b22f-image-registry-private-configuration\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:10:19.425370 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:19.425360    2576 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-tls\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:10:19.425370 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:19.425370    2576 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-bound-sa-token\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:10:19.425370 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:19.425379    2576 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9cb71e8f-b673-49d6-bce3-2b205b80b22f-ca-trust-extracted\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:10:19.425700 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:19.425388    2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9cb71e8f-b673-49d6-bce3-2b205b80b22f-trusted-ca\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:10:19.425700 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:19.425396    2576 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9cb71e8f-b673-49d6-bce3-2b205b80b22f-registry-certificates\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:10:19.425700 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:19.425404    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lw4gn\" (UniqueName: \"kubernetes.io/projected/9cb71e8f-b673-49d6-bce3-2b205b80b22f-kube-api-access-lw4gn\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:10:19.425700 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:19.425414    2576 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9cb71e8f-b673-49d6-bce3-2b205b80b22f-installation-pull-secrets\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:10:20.188657 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:20.188616    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-557757c5cd-9l2fg" event={"ID":"9cb71e8f-b673-49d6-bce3-2b205b80b22f","Type":"ContainerDied","Data":"653711d0d675a56777ef4af3146089300c6dbf3482204a193226f6f7ed850db7"}
Sep 15 20:10:20.189066 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:20.188706    2576 scope.go:117] "RemoveContainer" containerID="03b7f5de8962492d4d8830f0ed29dedef67895179f350323e82afac5c279d174"
Sep 15 20:10:20.189066 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:20.188643    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-557757c5cd-9l2fg"
Sep 15 20:10:20.206803 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:20.206778    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-557757c5cd-9l2fg"]
Sep 15 20:10:20.211015 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:20.210911    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-557757c5cd-9l2fg"]
Sep 15 20:10:20.538001 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:20.537948    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:10:20.681723 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:20.681686    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:10:22.143217 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:22.143183    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cb71e8f-b673-49d6-bce3-2b205b80b22f" path="/var/lib/kubelet/pods/9cb71e8f-b673-49d6-bce3-2b205b80b22f/volumes"
Sep 15 20:10:22.225162 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:22.225129    2576 generic.go:356] "Generic (PLEG): container finished" podID="eaa018c5-5f77-486a-a95d-aa730c416c06" containerID="d0a43d789ea01dd18910ab29776a4155ebe0cae05adaf75c29c31078481481a3" exitCode=0
Sep 15 20:10:22.225304 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:22.225200    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jcrqk" event={"ID":"eaa018c5-5f77-486a-a95d-aa730c416c06","Type":"ContainerDied","Data":"d0a43d789ea01dd18910ab29776a4155ebe0cae05adaf75c29c31078481481a3"}
Sep 15 20:10:22.225531 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:22.225515    2576 scope.go:117] "RemoveContainer" containerID="d0a43d789ea01dd18910ab29776a4155ebe0cae05adaf75c29c31078481481a3"
Sep 15 20:10:23.231106 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:23.231075    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jcrqk" event={"ID":"eaa018c5-5f77-486a-a95d-aa730c416c06","Type":"ContainerStarted","Data":"e9fa936c2f839ee75288d21fdb535bc53be2a42f16654942c008ceeb7629edb4"}
Sep 15 20:10:42.808141 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:42.808051    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:10:42.823544 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:42.823518    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:10:43.324934 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:43.324894    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:10:52.353867 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:52.353829    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:10:52.405242 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:52.405214    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:10:59.751848 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:59.751812    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-metrics-certs\") pod \"network-metrics-daemon-59mfs\" (UID: \"3b8d9342-6574-4d0d-a326-0299eb3b4c5d\") " pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:10:59.754118 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:10:59.754099    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b8d9342-6574-4d0d-a326-0299eb3b4c5d-metrics-certs\") pod \"network-metrics-daemon-59mfs\" (UID: \"3b8d9342-6574-4d0d-a326-0299eb3b4c5d\") " pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:11:00.040750 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:00.040676    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l9zsw\""
Sep 15 20:11:00.048667 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:00.048646    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59mfs"
Sep 15 20:11:00.320282 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:00.320258    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-59mfs"]
Sep 15 20:11:00.322886 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:11:00.322864    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b8d9342_6574_4d0d_a326_0299eb3b4c5d.slice/crio-af273dec57bd584b39f11e321f8ffe17d23ac18f49f1433509dfa9003a1a232d WatchSource:0}: Error finding container af273dec57bd584b39f11e321f8ffe17d23ac18f49f1433509dfa9003a1a232d: Status 404 returned error can't find the container with id af273dec57bd584b39f11e321f8ffe17d23ac18f49f1433509dfa9003a1a232d
Sep 15 20:11:00.377503 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:00.377471    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-59mfs" event={"ID":"3b8d9342-6574-4d0d-a326-0299eb3b4c5d","Type":"ContainerStarted","Data":"af273dec57bd584b39f11e321f8ffe17d23ac18f49f1433509dfa9003a1a232d"}
Sep 15 20:11:00.869877 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:00.869838    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 20:11:00.870304 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:00.870237    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="21554748-1332-4b7e-a9ff-df0b33c48b19" containerName="kube-rbac-proxy" containerID="cri-o://5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20" gracePeriod=600
Sep 15 20:11:00.870304 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:00.870276    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="21554748-1332-4b7e-a9ff-df0b33c48b19" containerName="kube-rbac-proxy-web" containerID="cri-o://e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7" gracePeriod=600
Sep 15 20:11:00.870304 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:00.870274    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="21554748-1332-4b7e-a9ff-df0b33c48b19" containerName="thanos-sidecar" containerID="cri-o://1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c" gracePeriod=600
Sep 15 20:11:00.870493 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:00.870223    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="21554748-1332-4b7e-a9ff-df0b33c48b19" containerName="prometheus" containerID="cri-o://7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1" gracePeriod=600
Sep 15 20:11:00.870493 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:00.870325    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="21554748-1332-4b7e-a9ff-df0b33c48b19" containerName="config-reloader" containerID="cri-o://9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d" gracePeriod=600
Sep 15 20:11:00.870493 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:00.870253    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="21554748-1332-4b7e-a9ff-df0b33c48b19" containerName="kube-rbac-proxy-thanos" containerID="cri-o://1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7" gracePeriod=600
Sep 15 20:11:01.132965 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.132938    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.263765 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.263732    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-configmap-kubelet-serving-ca-bundle\") pod \"21554748-1332-4b7e-a9ff-df0b33c48b19\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") "
Sep 15 20:11:01.263945 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.263776    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-prometheus-k8s-rulefiles-1\") pod \"21554748-1332-4b7e-a9ff-df0b33c48b19\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") "
Sep 15 20:11:01.263945 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.263798    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"21554748-1332-4b7e-a9ff-df0b33c48b19\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") "
Sep 15 20:11:01.263945 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.263831    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-config\") pod \"21554748-1332-4b7e-a9ff-df0b33c48b19\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") "
Sep 15 20:11:01.263945 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.263849    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-kube-rbac-proxy\") pod \"21554748-1332-4b7e-a9ff-df0b33c48b19\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") "
Sep 15 20:11:01.263945 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.263869    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-grpc-tls\") pod \"21554748-1332-4b7e-a9ff-df0b33c48b19\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") "
Sep 15 20:11:01.263945 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.263893    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/21554748-1332-4b7e-a9ff-df0b33c48b19-prometheus-k8s-db\") pod \"21554748-1332-4b7e-a9ff-df0b33c48b19\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") "
Sep 15 20:11:01.263945 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.263935    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"21554748-1332-4b7e-a9ff-df0b33c48b19\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") "
Sep 15 20:11:01.264295 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.263966    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-prometheus-k8s-rulefiles-2\") pod \"21554748-1332-4b7e-a9ff-df0b33c48b19\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") "
Sep 15 20:11:01.264295 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.263993    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-prometheus-k8s-tls\") pod \"21554748-1332-4b7e-a9ff-df0b33c48b19\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") "
Sep 15 20:11:01.264295 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.264016    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-configmap-serving-certs-ca-bundle\") pod \"21554748-1332-4b7e-a9ff-df0b33c48b19\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") "
Sep 15 20:11:01.264295 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.264048    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xg8gr\" (UniqueName: \"kubernetes.io/projected/21554748-1332-4b7e-a9ff-df0b33c48b19-kube-api-access-xg8gr\") pod \"21554748-1332-4b7e-a9ff-df0b33c48b19\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") "
Sep 15 20:11:01.264295 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.264070    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-metrics-client-certs\") pod \"21554748-1332-4b7e-a9ff-df0b33c48b19\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") "
Sep 15 20:11:01.264295 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.264084    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/21554748-1332-4b7e-a9ff-df0b33c48b19-tls-assets\") pod \"21554748-1332-4b7e-a9ff-df0b33c48b19\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") "
Sep 15 20:11:01.264295 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.264111    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-thanos-prometheus-http-client-file\") pod \"21554748-1332-4b7e-a9ff-df0b33c48b19\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") "
Sep 15 20:11:01.264295 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.264149    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-prometheus-k8s-rulefiles-0\") pod \"21554748-1332-4b7e-a9ff-df0b33c48b19\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") "
Sep 15 20:11:01.264295 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.264179    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-web-config\") pod \"21554748-1332-4b7e-a9ff-df0b33c48b19\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") "
Sep 15 20:11:01.264295 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.264204    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-configmap-metrics-client-ca\") pod \"21554748-1332-4b7e-a9ff-df0b33c48b19\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") "
Sep 15 20:11:01.264295 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.264244    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-prometheus-trusted-ca-bundle\") pod \"21554748-1332-4b7e-a9ff-df0b33c48b19\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") "
Sep 15 20:11:01.264295 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.264273    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/21554748-1332-4b7e-a9ff-df0b33c48b19-config-out\") pod \"21554748-1332-4b7e-a9ff-df0b33c48b19\" (UID: \"21554748-1332-4b7e-a9ff-df0b33c48b19\") "
Sep 15 20:11:01.265437 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.264902    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "21554748-1332-4b7e-a9ff-df0b33c48b19" (UID: "21554748-1332-4b7e-a9ff-df0b33c48b19"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:11:01.265437 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.265310    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "21554748-1332-4b7e-a9ff-df0b33c48b19" (UID: "21554748-1332-4b7e-a9ff-df0b33c48b19"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:11:01.265552 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.265461    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21554748-1332-4b7e-a9ff-df0b33c48b19-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "21554748-1332-4b7e-a9ff-df0b33c48b19" (UID: "21554748-1332-4b7e-a9ff-df0b33c48b19"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:11:01.267049 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.267020    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21554748-1332-4b7e-a9ff-df0b33c48b19-config-out" (OuterVolumeSpecName: "config-out") pod "21554748-1332-4b7e-a9ff-df0b33c48b19" (UID: "21554748-1332-4b7e-a9ff-df0b33c48b19"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:11:01.267409 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.267240    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "21554748-1332-4b7e-a9ff-df0b33c48b19" (UID: "21554748-1332-4b7e-a9ff-df0b33c48b19"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:11:01.267660 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.267610    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "21554748-1332-4b7e-a9ff-df0b33c48b19" (UID: "21554748-1332-4b7e-a9ff-df0b33c48b19"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:11:01.267756 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.267733    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "21554748-1332-4b7e-a9ff-df0b33c48b19" (UID: "21554748-1332-4b7e-a9ff-df0b33c48b19"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:11:01.268385 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.268126    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "21554748-1332-4b7e-a9ff-df0b33c48b19" (UID: "21554748-1332-4b7e-a9ff-df0b33c48b19"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:11:01.268385 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.268223    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "21554748-1332-4b7e-a9ff-df0b33c48b19" (UID: "21554748-1332-4b7e-a9ff-df0b33c48b19"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:11:01.268532 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.268436    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "21554748-1332-4b7e-a9ff-df0b33c48b19" (UID: "21554748-1332-4b7e-a9ff-df0b33c48b19"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:11:01.268532 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.268475    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-config" (OuterVolumeSpecName: "config") pod "21554748-1332-4b7e-a9ff-df0b33c48b19" (UID: "21554748-1332-4b7e-a9ff-df0b33c48b19"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:11:01.269084 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.268998    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "21554748-1332-4b7e-a9ff-df0b33c48b19" (UID: "21554748-1332-4b7e-a9ff-df0b33c48b19"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:11:01.269084 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.269031    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "21554748-1332-4b7e-a9ff-df0b33c48b19" (UID: "21554748-1332-4b7e-a9ff-df0b33c48b19"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:11:01.269319 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.269298    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "21554748-1332-4b7e-a9ff-df0b33c48b19" (UID: "21554748-1332-4b7e-a9ff-df0b33c48b19"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:11:01.269657 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.269621    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "21554748-1332-4b7e-a9ff-df0b33c48b19" (UID: "21554748-1332-4b7e-a9ff-df0b33c48b19"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:11:01.271126 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.271098    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "21554748-1332-4b7e-a9ff-df0b33c48b19" (UID: "21554748-1332-4b7e-a9ff-df0b33c48b19"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:11:01.271896 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.271853    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "21554748-1332-4b7e-a9ff-df0b33c48b19" (UID: "21554748-1332-4b7e-a9ff-df0b33c48b19"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:11:01.272011 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.271958    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21554748-1332-4b7e-a9ff-df0b33c48b19-kube-api-access-xg8gr" (OuterVolumeSpecName: "kube-api-access-xg8gr") pod "21554748-1332-4b7e-a9ff-df0b33c48b19" (UID: "21554748-1332-4b7e-a9ff-df0b33c48b19"). InnerVolumeSpecName "kube-api-access-xg8gr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:11:01.273152 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.273123    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21554748-1332-4b7e-a9ff-df0b33c48b19-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "21554748-1332-4b7e-a9ff-df0b33c48b19" (UID: "21554748-1332-4b7e-a9ff-df0b33c48b19"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:11:01.284236 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.284207    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-web-config" (OuterVolumeSpecName: "web-config") pod "21554748-1332-4b7e-a9ff-df0b33c48b19" (UID: "21554748-1332-4b7e-a9ff-df0b33c48b19"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:11:01.365856 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.365826    2576 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:11:01.365856 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.365855    2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:11:01.366078 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.365872    2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:11:01.366078 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.365885    2576 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-config\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:11:01.366078 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.365899    2576 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-kube-rbac-proxy\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:11:01.366078 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.365911    2576 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-grpc-tls\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:11:01.366078 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.365937    2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/21554748-1332-4b7e-a9ff-df0b33c48b19-prometheus-k8s-db\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:11:01.366078 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.365949    2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:11:01.366078 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.365961    2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:11:01.366078 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.365974    2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-prometheus-k8s-tls\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:11:01.366078 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.365987    2576 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:11:01.366078 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.366000    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xg8gr\" (UniqueName: \"kubernetes.io/projected/21554748-1332-4b7e-a9ff-df0b33c48b19-kube-api-access-xg8gr\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:11:01.366078 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.366012    2576 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-secret-metrics-client-certs\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:11:01.366078 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.366024    2576 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/21554748-1332-4b7e-a9ff-df0b33c48b19-tls-assets\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:11:01.366078 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.366038    2576 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-thanos-prometheus-http-client-file\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:11:01.366078 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.366054    2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:11:01.366078 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.366067    2576 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/21554748-1332-4b7e-a9ff-df0b33c48b19-web-config\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:11:01.366078 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.366079    2576 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-configmap-metrics-client-ca\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:11:01.366579 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.366093    2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21554748-1332-4b7e-a9ff-df0b33c48b19-prometheus-trusted-ca-bundle\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:11:01.366579 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.366107    2576 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/21554748-1332-4b7e-a9ff-df0b33c48b19-config-out\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:11:01.384765 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.384699    2576 generic.go:356] "Generic (PLEG): container finished" podID="21554748-1332-4b7e-a9ff-df0b33c48b19" containerID="1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7" exitCode=0
Sep 15 20:11:01.384765 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.384725    2576 generic.go:356] "Generic (PLEG): container finished" podID="21554748-1332-4b7e-a9ff-df0b33c48b19" containerID="5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20" exitCode=0
Sep 15 20:11:01.384765 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.384732    2576 generic.go:356] "Generic (PLEG): container finished" podID="21554748-1332-4b7e-a9ff-df0b33c48b19" containerID="e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7" exitCode=0
Sep 15 20:11:01.384765 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.384739    2576 generic.go:356] "Generic (PLEG): container finished" podID="21554748-1332-4b7e-a9ff-df0b33c48b19" containerID="1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c" exitCode=0
Sep 15 20:11:01.384765 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.384744    2576 generic.go:356] "Generic (PLEG): container finished" podID="21554748-1332-4b7e-a9ff-df0b33c48b19" containerID="9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d" exitCode=0
Sep 15 20:11:01.384765 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.384750    2576 generic.go:356] "Generic (PLEG): container finished" podID="21554748-1332-4b7e-a9ff-df0b33c48b19" containerID="7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1" exitCode=0
Sep 15 20:11:01.385185 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.384772    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.385185 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.384779    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"21554748-1332-4b7e-a9ff-df0b33c48b19","Type":"ContainerDied","Data":"1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7"}
Sep 15 20:11:01.385185 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.384870    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"21554748-1332-4b7e-a9ff-df0b33c48b19","Type":"ContainerDied","Data":"5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20"}
Sep 15 20:11:01.385185 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.384915    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"21554748-1332-4b7e-a9ff-df0b33c48b19","Type":"ContainerDied","Data":"e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7"}
Sep 15 20:11:01.385185 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.384940    2576 scope.go:117] "RemoveContainer" containerID="1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7"
Sep 15 20:11:01.385185 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.384976    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"21554748-1332-4b7e-a9ff-df0b33c48b19","Type":"ContainerDied","Data":"1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c"}
Sep 15 20:11:01.385185 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.385032    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"21554748-1332-4b7e-a9ff-df0b33c48b19","Type":"ContainerDied","Data":"9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d"}
Sep 15 20:11:01.385185 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.385074    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"21554748-1332-4b7e-a9ff-df0b33c48b19","Type":"ContainerDied","Data":"7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1"}
Sep 15 20:11:01.385185 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.385115    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"21554748-1332-4b7e-a9ff-df0b33c48b19","Type":"ContainerDied","Data":"41598c076e03653b2133a383f3e5f72e74ca07c4e5a2f6cdec7240405ca65877"}
Sep 15 20:11:01.393064 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.393045    2576 scope.go:117] "RemoveContainer" containerID="5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20"
Sep 15 20:11:01.400260 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.400237    2576 scope.go:117] "RemoveContainer" containerID="e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7"
Sep 15 20:11:01.408505 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.408486    2576 scope.go:117] "RemoveContainer" containerID="1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c"
Sep 15 20:11:01.416890 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.416867    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 20:11:01.419287 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.419265    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 20:11:01.420615 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.420596    2576 scope.go:117] "RemoveContainer" containerID="9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d"
Sep 15 20:11:01.429022 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.428998    2576 scope.go:117] "RemoveContainer" containerID="7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1"
Sep 15 20:11:01.437788 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.437771    2576 scope.go:117] "RemoveContainer" containerID="7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992"
Sep 15 20:11:01.442131 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.442111    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 20:11:01.442490 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.442475    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cb71e8f-b673-49d6-bce3-2b205b80b22f" containerName="registry"
Sep 15 20:11:01.442490 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.442492    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cb71e8f-b673-49d6-bce3-2b205b80b22f" containerName="registry"
Sep 15 20:11:01.442648 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.442506    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21554748-1332-4b7e-a9ff-df0b33c48b19" containerName="kube-rbac-proxy-thanos"
Sep 15 20:11:01.442648 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.442515    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="21554748-1332-4b7e-a9ff-df0b33c48b19" containerName="kube-rbac-proxy-thanos"
Sep 15 20:11:01.442648 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.442533    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21554748-1332-4b7e-a9ff-df0b33c48b19" containerName="config-reloader"
Sep 15 20:11:01.442648 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.442541    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="21554748-1332-4b7e-a9ff-df0b33c48b19" containerName="config-reloader"
Sep 15 20:11:01.442648 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.442553    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21554748-1332-4b7e-a9ff-df0b33c48b19" containerName="kube-rbac-proxy"
Sep 15 20:11:01.442648 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.442562    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="21554748-1332-4b7e-a9ff-df0b33c48b19" containerName="kube-rbac-proxy"
Sep 15 20:11:01.442648 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.442576    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21554748-1332-4b7e-a9ff-df0b33c48b19" containerName="init-config-reloader"
Sep 15 20:11:01.442648 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.442584    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="21554748-1332-4b7e-a9ff-df0b33c48b19" containerName="init-config-reloader"
Sep 15 20:11:01.442648 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.442594    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21554748-1332-4b7e-a9ff-df0b33c48b19" containerName="thanos-sidecar"
Sep 15 20:11:01.442648 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.442602    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="21554748-1332-4b7e-a9ff-df0b33c48b19" containerName="thanos-sidecar"
Sep 15 20:11:01.442648 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.442612    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21554748-1332-4b7e-a9ff-df0b33c48b19" containerName="prometheus"
Sep 15 20:11:01.442648 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.442621    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="21554748-1332-4b7e-a9ff-df0b33c48b19" containerName="prometheus"
Sep 15 20:11:01.442648 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.442631    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21554748-1332-4b7e-a9ff-df0b33c48b19" containerName="kube-rbac-proxy-web"
Sep 15 20:11:01.442648 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.442640    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="21554748-1332-4b7e-a9ff-df0b33c48b19" containerName="kube-rbac-proxy-web"
Sep 15 20:11:01.443339 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.442713    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="21554748-1332-4b7e-a9ff-df0b33c48b19" containerName="config-reloader"
Sep 15 20:11:01.443339 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.442725    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="21554748-1332-4b7e-a9ff-df0b33c48b19" containerName="kube-rbac-proxy"
Sep 15 20:11:01.443339 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.442735    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="21554748-1332-4b7e-a9ff-df0b33c48b19" containerName="thanos-sidecar"
Sep 15 20:11:01.443339 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.442744    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="21554748-1332-4b7e-a9ff-df0b33c48b19" containerName="kube-rbac-proxy-web"
Sep 15 20:11:01.443339 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.442755    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cb71e8f-b673-49d6-bce3-2b205b80b22f" containerName="registry"
Sep 15 20:11:01.443339 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.442762    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="21554748-1332-4b7e-a9ff-df0b33c48b19" containerName="kube-rbac-proxy-thanos"
Sep 15 20:11:01.443339 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.442773    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="21554748-1332-4b7e-a9ff-df0b33c48b19" containerName="prometheus"
Sep 15 20:11:01.446606 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.446585    2576 scope.go:117] "RemoveContainer" containerID="1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7"
Sep 15 20:11:01.447027 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:11:01.447003    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7\": container with ID starting with 1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7 not found: ID does not exist" containerID="1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7"
Sep 15 20:11:01.447115 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.447039    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7"} err="failed to get container status \"1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7\": rpc error: code = NotFound desc = could not find container \"1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7\": container with ID starting with 1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7 not found: ID does not exist"
Sep 15 20:11:01.447115 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.447082    2576 scope.go:117] "RemoveContainer" containerID="5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20"
Sep 15 20:11:01.447510 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:11:01.447471    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20\": container with ID starting with 5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20 not found: ID does not exist" containerID="5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20"
Sep 15 20:11:01.447622 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.447518    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20"} err="failed to get container status \"5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20\": rpc error: code = NotFound desc = could not find container \"5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20\": container with ID starting with 5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20 not found: ID does not exist"
Sep 15 20:11:01.447622 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.447541    2576 scope.go:117] "RemoveContainer" containerID="e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7"
Sep 15 20:11:01.447850 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:11:01.447823    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7\": container with ID starting with e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7 not found: ID does not exist" containerID="e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7"
Sep 15 20:11:01.448037 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.447857    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7"} err="failed to get container status \"e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7\": rpc error: code = NotFound desc = could not find container \"e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7\": container with ID starting with e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7 not found: ID does not exist"
Sep 15 20:11:01.448037 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.447878    2576 scope.go:117] "RemoveContainer" containerID="1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c"
Sep 15 20:11:01.448261 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:11:01.448233    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c\": container with ID starting with 1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c not found: ID does not exist" containerID="1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c"
Sep 15 20:11:01.448318 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.448263    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c"} err="failed to get container status \"1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c\": rpc error: code = NotFound desc = could not find container \"1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c\": container with ID starting with 1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c not found: ID does not exist"
Sep 15 20:11:01.448318 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.448282    2576 scope.go:117] "RemoveContainer" containerID="9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d"
Sep 15 20:11:01.448568 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:11:01.448553    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d\": container with ID starting with 9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d not found: ID does not exist" containerID="9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d"
Sep 15 20:11:01.448621 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.448571    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d"} err="failed to get container status \"9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d\": rpc error: code = NotFound desc = could not find container \"9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d\": container with ID starting with 9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d not found: ID does not exist"
Sep 15 20:11:01.448621 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.448583    2576 scope.go:117] "RemoveContainer" containerID="7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1"
Sep 15 20:11:01.448755 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.448737    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.448815 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:11:01.448796    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1\": container with ID starting with 7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1 not found: ID does not exist" containerID="7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1"
Sep 15 20:11:01.448870 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.448824    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1"} err="failed to get container status \"7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1\": rpc error: code = NotFound desc = could not find container \"7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1\": container with ID starting with 7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1 not found: ID does not exist"
Sep 15 20:11:01.448870 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.448845    2576 scope.go:117] "RemoveContainer" containerID="7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992"
Sep 15 20:11:01.449546 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:11:01.449518    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992\": container with ID starting with 7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992 not found: ID does not exist" containerID="7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992"
Sep 15 20:11:01.449634 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.449556    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992"} err="failed to get container status \"7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992\": rpc error: code = NotFound desc = could not find container \"7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992\": container with ID starting with 7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992 not found: ID does not exist"
Sep 15 20:11:01.449634 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.449577    2576 scope.go:117] "RemoveContainer" containerID="1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7"
Sep 15 20:11:01.450200 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.450168    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7"} err="failed to get container status \"1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7\": rpc error: code = NotFound desc = could not find container \"1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7\": container with ID starting with 1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7 not found: ID does not exist"
Sep 15 20:11:01.450200 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.450191    2576 scope.go:117] "RemoveContainer" containerID="5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20"
Sep 15 20:11:01.450442 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.450420    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20"} err="failed to get container status \"5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20\": rpc error: code = NotFound desc = could not find container \"5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20\": container with ID starting with 5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20 not found: ID does not exist"
Sep 15 20:11:01.450516 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.450444    2576 scope.go:117] "RemoveContainer" containerID="e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7"
Sep 15 20:11:01.450680 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.450663    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7"} err="failed to get container status \"e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7\": rpc error: code = NotFound desc = could not find container \"e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7\": container with ID starting with e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7 not found: ID does not exist"
Sep 15 20:11:01.450734 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.450683    2576 scope.go:117] "RemoveContainer" containerID="1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c"
Sep 15 20:11:01.451016 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.450986    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c"} err="failed to get container status \"1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c\": rpc error: code = NotFound desc = could not find container \"1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c\": container with ID starting with 1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c not found: ID does not exist"
Sep 15 20:11:01.451091 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.451037    2576 scope.go:117] "RemoveContainer" containerID="9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d"
Sep 15 20:11:01.451442 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.451391    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6h4m3k1d24rt4\""
Sep 15 20:11:01.451442 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.451422    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d"} err="failed to get container status \"9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d\": rpc error: code = NotFound desc = could not find container \"9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d\": container with ID starting with 9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d not found: ID does not exist"
Sep 15 20:11:01.451614 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.451450    2576 scope.go:117] "RemoveContainer" containerID="7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1"
Sep 15 20:11:01.451614 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.451455    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 15 20:11:01.451614 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.451467    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 15 20:11:01.451614 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.451513    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 15 20:11:01.451614 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.451560    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 15 20:11:01.451614 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.451609    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 15 20:11:01.451967 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.451812    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1"} err="failed to get container status \"7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1\": rpc error: code = NotFound desc = could not find container \"7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1\": container with ID starting with 7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1 not found: ID does not exist"
Sep 15 20:11:01.451967 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.451834    2576 scope.go:117] "RemoveContainer" containerID="7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992"
Sep 15 20:11:01.451967 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.451899    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 15 20:11:01.452123 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.451976    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 15 20:11:01.452123 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.451987    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 15 20:11:01.452123 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.452061    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 15 20:11:01.452123 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.451903    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 15 20:11:01.452324 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.452177    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-s6d2x\""
Sep 15 20:11:01.452377 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.452335    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 15 20:11:01.452377 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.452348    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 15 20:11:01.452377 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.452344    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992"} err="failed to get container status \"7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992\": rpc error: code = NotFound desc = could not find container \"7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992\": container with ID starting with 7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992 not found: ID does not exist"
Sep 15 20:11:01.452530 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.452382    2576 scope.go:117] "RemoveContainer" containerID="1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7"
Sep 15 20:11:01.452530 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.452438    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 15 20:11:01.452696 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.452667    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7"} err="failed to get container status \"1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7\": rpc error: code = NotFound desc = could not find container \"1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7\": container with ID starting with 1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7 not found: ID does not exist"
Sep 15 20:11:01.452696 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.452695    2576 scope.go:117] "RemoveContainer" containerID="5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20"
Sep 15 20:11:01.453375 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.453349    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20"} err="failed to get container status \"5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20\": rpc error: code = NotFound desc = could not find container \"5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20\": container with ID starting with 5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20 not found: ID does not exist"
Sep 15 20:11:01.453462 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.453376    2576 scope.go:117] "RemoveContainer" containerID="e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7"
Sep 15 20:11:01.453738 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.453711    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7"} err="failed to get container status \"e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7\": rpc error: code = NotFound desc = could not find container \"e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7\": container with ID starting with e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7 not found: ID does not exist"
Sep 15 20:11:01.453842 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.453740    2576 scope.go:117] "RemoveContainer" containerID="1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c"
Sep 15 20:11:01.454420 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.454084    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c"} err="failed to get container status \"1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c\": rpc error: code = NotFound desc = could not find container \"1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c\": container with ID starting with 1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c not found: ID does not exist"
Sep 15 20:11:01.454420 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.454111    2576 scope.go:117] "RemoveContainer" containerID="9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d"
Sep 15 20:11:01.455803 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.455598    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 15 20:11:01.457110 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.457050    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d"} err="failed to get container status \"9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d\": rpc error: code = NotFound desc = could not find container \"9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d\": container with ID starting with 9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d not found: ID does not exist"
Sep 15 20:11:01.457110 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.457078    2576 scope.go:117] "RemoveContainer" containerID="7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1"
Sep 15 20:11:01.457455 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.457426    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1"} err="failed to get container status \"7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1\": rpc error: code = NotFound desc = could not find container \"7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1\": container with ID starting with 7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1 not found: ID does not exist"
Sep 15 20:11:01.457455 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.457454    2576 scope.go:117] "RemoveContainer" containerID="7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992"
Sep 15 20:11:01.457785 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.457758    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992"} err="failed to get container status \"7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992\": rpc error: code = NotFound desc = could not find container \"7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992\": container with ID starting with 7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992 not found: ID does not exist"
Sep 15 20:11:01.457875 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.457788    2576 scope.go:117] "RemoveContainer" containerID="1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7"
Sep 15 20:11:01.458211 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.458185    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7"} err="failed to get container status \"1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7\": rpc error: code = NotFound desc = could not find container \"1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7\": container with ID starting with 1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7 not found: ID does not exist"
Sep 15 20:11:01.458313 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.458211    2576 scope.go:117] "RemoveContainer" containerID="5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20"
Sep 15 20:11:01.459032 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.458805    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20"} err="failed to get container status \"5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20\": rpc error: code = NotFound desc = could not find container \"5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20\": container with ID starting with 5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20 not found: ID does not exist"
Sep 15 20:11:01.459032 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.458831    2576 scope.go:117] "RemoveContainer" containerID="e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7"
Sep 15 20:11:01.459192 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.459165    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7"} err="failed to get container status \"e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7\": rpc error: code = NotFound desc = could not find container \"e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7\": container with ID starting with e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7 not found: ID does not exist"
Sep 15 20:11:01.459253 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.459191    2576 scope.go:117] "RemoveContainer" containerID="1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c"
Sep 15 20:11:01.459555 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.459528    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c"} err="failed to get container status \"1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c\": rpc error: code = NotFound desc = could not find container \"1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c\": container with ID starting with 1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c not found: ID does not exist"
Sep 15 20:11:01.459555 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.459555    2576 scope.go:117] "RemoveContainer" containerID="9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d"
Sep 15 20:11:01.459738 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.459719    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 15 20:11:01.459838 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.459814    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d"} err="failed to get container status \"9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d\": rpc error: code = NotFound desc = could not find container \"9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d\": container with ID starting with 9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d not found: ID does not exist"
Sep 15 20:11:01.459901 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.459841    2576 scope.go:117] "RemoveContainer" containerID="7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1"
Sep 15 20:11:01.460193 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.460164    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1"} err="failed to get container status \"7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1\": rpc error: code = NotFound desc = could not find container \"7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1\": container with ID starting with 7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1 not found: ID does not exist"
Sep 15 20:11:01.460193 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.460190    2576 scope.go:117] "RemoveContainer" containerID="7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992"
Sep 15 20:11:01.460449 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.460425    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992"} err="failed to get container status \"7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992\": rpc error: code = NotFound desc = could not find container \"7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992\": container with ID starting with 7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992 not found: ID does not exist"
Sep 15 20:11:01.460529 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.460451    2576 scope.go:117] "RemoveContainer" containerID="1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7"
Sep 15 20:11:01.460750 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.460727    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7"} err="failed to get container status \"1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7\": rpc error: code = NotFound desc = could not find container \"1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7\": container with ID starting with 1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7 not found: ID does not exist"
Sep 15 20:11:01.460750 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.460749    2576 scope.go:117] "RemoveContainer" containerID="5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20"
Sep 15 20:11:01.461143 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.461123    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20"} err="failed to get container status \"5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20\": rpc error: code = NotFound desc = could not find container \"5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20\": container with ID starting with 5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20 not found: ID does not exist"
Sep 15 20:11:01.461143 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.461143    2576 scope.go:117] "RemoveContainer" containerID="e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7"
Sep 15 20:11:01.461493 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.461442    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7"} err="failed to get container status \"e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7\": rpc error: code = NotFound desc = could not find container \"e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7\": container with ID starting with e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7 not found: ID does not exist"
Sep 15 20:11:01.461493 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.461469    2576 scope.go:117] "RemoveContainer" containerID="1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c"
Sep 15 20:11:01.462035 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.461847    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c"} err="failed to get container status \"1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c\": rpc error: code = NotFound desc = could not find container \"1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c\": container with ID starting with 1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c not found: ID does not exist"
Sep 15 20:11:01.462035 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.461873    2576 scope.go:117] "RemoveContainer" containerID="9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d"
Sep 15 20:11:01.462861 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.462168    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d"} err="failed to get container status \"9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d\": rpc error: code = NotFound desc = could not find container \"9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d\": container with ID starting with 9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d not found: ID does not exist"
Sep 15 20:11:01.462861 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.462189    2576 scope.go:117] "RemoveContainer" containerID="7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1"
Sep 15 20:11:01.462861 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.462604    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1"} err="failed to get container status \"7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1\": rpc error: code = NotFound desc = could not find container \"7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1\": container with ID starting with 7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1 not found: ID does not exist"
Sep 15 20:11:01.462861 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.462664    2576 scope.go:117] "RemoveContainer" containerID="7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992"
Sep 15 20:11:01.463094 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.463037    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992"} err="failed to get container status \"7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992\": rpc error: code = NotFound desc = could not find container \"7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992\": container with ID starting with 7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992 not found: ID does not exist"
Sep 15 20:11:01.463094 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.463060    2576 scope.go:117] "RemoveContainer" containerID="1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7"
Sep 15 20:11:01.463367 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.463330    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7"} err="failed to get container status \"1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7\": rpc error: code = NotFound desc = could not find container \"1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7\": container with ID starting with 1006fe0d11cfec53c36c6e039788228e2a5fd209ab4491deb6a7868488a0d4b7 not found: ID does not exist"
Sep 15 20:11:01.463367 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.463357    2576 scope.go:117] "RemoveContainer" containerID="5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20"
Sep 15 20:11:01.463604 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.463584    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20"} err="failed to get container status \"5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20\": rpc error: code = NotFound desc = could not find container \"5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20\": container with ID starting with 5e2d2dd77b260957b16de274faae45428e0003115dae6f6abf25aab7fc24fd20 not found: ID does not exist"
Sep 15 20:11:01.463604 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.463602    2576 scope.go:117] "RemoveContainer" containerID="e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7"
Sep 15 20:11:01.463852 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.463827    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7"} err="failed to get container status \"e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7\": rpc error: code = NotFound desc = could not find container \"e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7\": container with ID starting with e24eeb9b3200fe3aa581844632502daa97b748317d7d33c0c7552c6da60d09e7 not found: ID does not exist"
Sep 15 20:11:01.463968 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.463853    2576 scope.go:117] "RemoveContainer" containerID="1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c"
Sep 15 20:11:01.464330 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.464294    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c"} err="failed to get container status \"1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c\": rpc error: code = NotFound desc = could not find container \"1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c\": container with ID starting with 1af76390f9378588f2aad197767fea0af1767ae68092d028e0066042ba0d490c not found: ID does not exist"
Sep 15 20:11:01.464330 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.464322    2576 scope.go:117] "RemoveContainer" containerID="9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d"
Sep 15 20:11:01.464582 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.464562    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d"} err="failed to get container status \"9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d\": rpc error: code = NotFound desc = could not find container \"9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d\": container with ID starting with 9db76c00b5fd7700466c5414b1bef687d4a44a60c12b1fa39b51a3808829c05d not found: ID does not exist"
Sep 15 20:11:01.464582 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.464583    2576 scope.go:117] "RemoveContainer" containerID="7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1"
Sep 15 20:11:01.464886 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.464844    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 20:11:01.465031 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.464990    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1"} err="failed to get container status \"7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1\": rpc error: code = NotFound desc = could not find container \"7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1\": container with ID starting with 7be3a08ef78e2ad996a06394bd1e764a3295b1c0270fd040d7674d2649c129b1 not found: ID does not exist"
Sep 15 20:11:01.465031 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.465010    2576 scope.go:117] "RemoveContainer" containerID="7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992"
Sep 15 20:11:01.465639 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.465600    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992"} err="failed to get container status \"7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992\": rpc error: code = NotFound desc = could not find container \"7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992\": container with ID starting with 7a9ff6d13c08188a6cc846e746c9e64c51eaab3f3cc4ab7427063bd538a71992 not found: ID does not exist"
Sep 15 20:11:01.567834 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.567795    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7850e2c8-449b-43b9-bdc3-896a75717f66-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.567834 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.567832    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7850e2c8-449b-43b9-bdc3-896a75717f66-config-out\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.568069 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.567847    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7850e2c8-449b-43b9-bdc3-896a75717f66-web-config\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.568069 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.567864    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7850e2c8-449b-43b9-bdc3-896a75717f66-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.568069 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.567881    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7850e2c8-449b-43b9-bdc3-896a75717f66-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.568069 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.567898    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7850e2c8-449b-43b9-bdc3-896a75717f66-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.568069 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.567917    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7850e2c8-449b-43b9-bdc3-896a75717f66-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.568069 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.567994    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7850e2c8-449b-43b9-bdc3-896a75717f66-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.568069 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.568037    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7850e2c8-449b-43b9-bdc3-896a75717f66-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.568069 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.568056    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6phch\" (UniqueName: \"kubernetes.io/projected/7850e2c8-449b-43b9-bdc3-896a75717f66-kube-api-access-6phch\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.568407 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.568074    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7850e2c8-449b-43b9-bdc3-896a75717f66-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.568407 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.568097    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7850e2c8-449b-43b9-bdc3-896a75717f66-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.568407 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.568117    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7850e2c8-449b-43b9-bdc3-896a75717f66-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.568407 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.568172    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7850e2c8-449b-43b9-bdc3-896a75717f66-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.568407 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.568210    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7850e2c8-449b-43b9-bdc3-896a75717f66-config\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.568407 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.568238    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7850e2c8-449b-43b9-bdc3-896a75717f66-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.568407 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.568262    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7850e2c8-449b-43b9-bdc3-896a75717f66-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.568407 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.568290    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7850e2c8-449b-43b9-bdc3-896a75717f66-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.568407 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.568315    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7850e2c8-449b-43b9-bdc3-896a75717f66-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.568407 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.568371    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7850e2c8-449b-43b9-bdc3-896a75717f66-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.669271 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.669185    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7850e2c8-449b-43b9-bdc3-896a75717f66-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.669271 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.669226    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7850e2c8-449b-43b9-bdc3-896a75717f66-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.669271 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.669261    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7850e2c8-449b-43b9-bdc3-896a75717f66-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.669518 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.669286    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7850e2c8-449b-43b9-bdc3-896a75717f66-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.669518 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.669334    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7850e2c8-449b-43b9-bdc3-896a75717f66-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.669518 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.669364    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6phch\" (UniqueName: \"kubernetes.io/projected/7850e2c8-449b-43b9-bdc3-896a75717f66-kube-api-access-6phch\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.669518 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.669390    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7850e2c8-449b-43b9-bdc3-896a75717f66-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.669518 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.669426    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7850e2c8-449b-43b9-bdc3-896a75717f66-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.669518 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.669455    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7850e2c8-449b-43b9-bdc3-896a75717f66-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.669518 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.669485    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7850e2c8-449b-43b9-bdc3-896a75717f66-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.669518 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.669517    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7850e2c8-449b-43b9-bdc3-896a75717f66-config\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.669903 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.669545    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7850e2c8-449b-43b9-bdc3-896a75717f66-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.669903 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.669566    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7850e2c8-449b-43b9-bdc3-896a75717f66-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.669903 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.669596    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7850e2c8-449b-43b9-bdc3-896a75717f66-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.669903 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.669622    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7850e2c8-449b-43b9-bdc3-896a75717f66-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.669903 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.669665    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7850e2c8-449b-43b9-bdc3-896a75717f66-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.669903 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.669703    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7850e2c8-449b-43b9-bdc3-896a75717f66-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.669903 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.669733    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7850e2c8-449b-43b9-bdc3-896a75717f66-config-out\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.669903 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.669756    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7850e2c8-449b-43b9-bdc3-896a75717f66-web-config\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.669903 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.669787    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7850e2c8-449b-43b9-bdc3-896a75717f66-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.670480 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.669906    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7850e2c8-449b-43b9-bdc3-896a75717f66-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.670480 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.669976    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7850e2c8-449b-43b9-bdc3-896a75717f66-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.670480 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.670317    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7850e2c8-449b-43b9-bdc3-896a75717f66-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.670620 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.670517    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7850e2c8-449b-43b9-bdc3-896a75717f66-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.670796 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.670771    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7850e2c8-449b-43b9-bdc3-896a75717f66-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.673863 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.672972    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7850e2c8-449b-43b9-bdc3-896a75717f66-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.673863 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.673567    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7850e2c8-449b-43b9-bdc3-896a75717f66-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.673863 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.673772    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7850e2c8-449b-43b9-bdc3-896a75717f66-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.675153 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.674709    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7850e2c8-449b-43b9-bdc3-896a75717f66-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.675153 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.675074    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7850e2c8-449b-43b9-bdc3-896a75717f66-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.675153 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.675112    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7850e2c8-449b-43b9-bdc3-896a75717f66-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.675346 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.675254    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7850e2c8-449b-43b9-bdc3-896a75717f66-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.675751 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.675696    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7850e2c8-449b-43b9-bdc3-896a75717f66-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.675935 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.675897    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7850e2c8-449b-43b9-bdc3-896a75717f66-config-out\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.676191 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.676149    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/7850e2c8-449b-43b9-bdc3-896a75717f66-config\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.676488 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.676468    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7850e2c8-449b-43b9-bdc3-896a75717f66-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.676645 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.676624    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7850e2c8-449b-43b9-bdc3-896a75717f66-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.677213 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.677193    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7850e2c8-449b-43b9-bdc3-896a75717f66-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.677736 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.677717    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7850e2c8-449b-43b9-bdc3-896a75717f66-web-config\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.678778 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.678756    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6phch\" (UniqueName: \"kubernetes.io/projected/7850e2c8-449b-43b9-bdc3-896a75717f66-kube-api-access-6phch\") pod \"prometheus-k8s-0\" (UID: \"7850e2c8-449b-43b9-bdc3-896a75717f66\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:01.761725 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:01.761678    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:02.142961 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:02.142916    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21554748-1332-4b7e-a9ff-df0b33c48b19" path="/var/lib/kubelet/pods/21554748-1332-4b7e-a9ff-df0b33c48b19/volumes"
Sep 15 20:11:02.159575 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:02.159475    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 20:11:02.161845 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:11:02.161822    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7850e2c8_449b_43b9_bdc3_896a75717f66.slice/crio-c63b271a8a8bf31bab6333057ac665e17c7f554f0fae92e0aa3541f044548152 WatchSource:0}: Error finding container c63b271a8a8bf31bab6333057ac665e17c7f554f0fae92e0aa3541f044548152: Status 404 returned error can't find the container with id c63b271a8a8bf31bab6333057ac665e17c7f554f0fae92e0aa3541f044548152
Sep 15 20:11:02.391008 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:02.390973    2576 generic.go:356] "Generic (PLEG): container finished" podID="7850e2c8-449b-43b9-bdc3-896a75717f66" containerID="b8aa80ad78d4d6db4397b3f0889319aa687dca90bd3934ccdb011b4c28f15e2b" exitCode=0
Sep 15 20:11:02.391182 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:02.391061    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7850e2c8-449b-43b9-bdc3-896a75717f66","Type":"ContainerDied","Data":"b8aa80ad78d4d6db4397b3f0889319aa687dca90bd3934ccdb011b4c28f15e2b"}
Sep 15 20:11:02.391182 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:02.391132    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7850e2c8-449b-43b9-bdc3-896a75717f66","Type":"ContainerStarted","Data":"c63b271a8a8bf31bab6333057ac665e17c7f554f0fae92e0aa3541f044548152"}
Sep 15 20:11:02.393898 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:02.393875    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-59mfs" event={"ID":"3b8d9342-6574-4d0d-a326-0299eb3b4c5d","Type":"ContainerStarted","Data":"137f53a06c8f2585043075d3efe2a69c06398e5c0ef24a0a68231cfc9de5431f"}
Sep 15 20:11:02.394017 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:02.393939    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-59mfs" event={"ID":"3b8d9342-6574-4d0d-a326-0299eb3b4c5d","Type":"ContainerStarted","Data":"30bf9beaed1b934a79b6de5f040a4673b5c0d7678ce2499093a52e7a2481260a"}
Sep 15 20:11:02.441709 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:02.441663    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-59mfs" podStartSLOduration=374.846809589 podStartE2EDuration="6m16.441646007s" totalImagesPullingTime="1.594836418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:11:02.440840654 +0000 UTC m=+376.929305130" watchObservedRunningTime="2026-09-15 20:11:02.441646007 +0000 UTC m=+376.930110483"
Sep 15 20:11:02.568278 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:02.568250    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:11:02.686974 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:02.686848    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:11:03.401341 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:03.401300    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7850e2c8-449b-43b9-bdc3-896a75717f66","Type":"ContainerStarted","Data":"d17a0e0d2f5d016df4507b4f4ec12209d41d1fe67c1f09e000b119d4c3a82040"}
Sep 15 20:11:03.401802 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:03.401415    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7850e2c8-449b-43b9-bdc3-896a75717f66","Type":"ContainerStarted","Data":"3b4657406c8d121398efe973cc3f5ab1420ff5681781df3fccc4ece7df787828"}
Sep 15 20:11:03.401802 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:03.401467    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7850e2c8-449b-43b9-bdc3-896a75717f66","Type":"ContainerStarted","Data":"e9bedaae919ce885e2d0165ae72a66eb47147acea828acf4949ab3ef5189adec"}
Sep 15 20:11:03.401802 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:03.401512    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7850e2c8-449b-43b9-bdc3-896a75717f66","Type":"ContainerStarted","Data":"756e293f09eceb7c3e2837be0690c39afe2b644d40dc6f7ef4d7bd89d9b4de75"}
Sep 15 20:11:03.401802 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:03.401548    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7850e2c8-449b-43b9-bdc3-896a75717f66","Type":"ContainerStarted","Data":"f9a12be629233d40c6b7f0e891aeeba6b5f46dd441408571f9ac2c88077dbacd"}
Sep 15 20:11:03.401802 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:03.401588    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7850e2c8-449b-43b9-bdc3-896a75717f66","Type":"ContainerStarted","Data":"7c84785793b7e2032e618e7e6d1038c9a268b8d14be49b85b7632c3b8f865b86"}
Sep 15 20:11:03.435523 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:03.435442    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.4177620109999998 podStartE2EDuration="2.435426518s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.664507ms" isStatefulPod=false podCreationTimestamp="2026-09-15 20:11:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:11:03.43459288 +0000 UTC m=+377.923057370" watchObservedRunningTime="2026-09-15 20:11:03.435426518 +0000 UTC m=+377.923891033"
Sep 15 20:11:04.256293 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:04.256257    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:11:04.322189 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:04.322162    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:11:06.762429 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:06.762393    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:11:10.494099 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:10.494054    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:11:10.558524 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:10.558498    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:11:20.416895 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:20.416862    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:11:20.475163 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:20.475133    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:11:20.550000 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:20.549973    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:11:20.649654 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:20.649625    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:11:31.576547 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:31.576512    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:11:31.680876 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:31.680849    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:11:31.739135 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:31.739110    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:11:37.132293 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:37.132258    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:11:54.384476 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:54.384438    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:11:54.483823 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:11:54.483793    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:12:01.762807 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:12:01.762765    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:12:01.778211 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:12:01.778187    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:12:02.652745 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:12:02.652713    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 20:12:21.254673 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:12:21.254641    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:12:22.074303 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:12:22.074266    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:12:25.049898 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:12:25.049859    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:12:25.150953 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:12:25.150900    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:12:27.134082 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:12:27.134046    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:12:27.295025 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:12:27.294991    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:12:29.502859 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:12:29.502822    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:12:29.664087 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:12:29.664057    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:12:31.875964 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:12:31.875914    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:12:31.970979 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:12:31.970945    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:12:32.029239 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:12:32.029206    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:12:36.600779 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:12:36.600745    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:12:36.862548 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:12:36.862478    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:12:37.156331 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:12:37.156259    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:12:37.271164 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:12:37.271138    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:12:43.255131 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:12:43.255096    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:12:43.355659 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:12:43.355628    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:12:48.508542 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:12:48.508508    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:12:48.566742 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:12:48.566717    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:12:50.551618 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:12:50.551585    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:12:53.402799 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:12:53.402760    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:12:53.520176 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:12:53.520144    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:12:55.300131 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:12:55.300096    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:03.218799 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:03.218767    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:03.322960 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:03.322905    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:03.853902 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:03.853872    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:04.301391 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:04.301359    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:04.394493 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:04.394468    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:04.449435 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:04.449406    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-75fcf566-t6w5p"]
Sep 15 20:13:04.452738 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:04.452723    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-t6w5p"
Sep 15 20:13:04.456377 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:04.456341    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 15 20:13:04.456501 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:04.456483    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 15 20:13:04.456734 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:04.456718    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-d6xtv\""
Sep 15 20:13:04.456734 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:04.456730    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 15 20:13:04.458168 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:04.458149    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 15 20:13:04.468644 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:04.468620    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-75fcf566-t6w5p"]
Sep 15 20:13:04.511511 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:04.511489    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f45205a-57e4-4819-931e-f3533ee73a92-tmp\") pod \"opendatahub-operator-controller-manager-75fcf566-t6w5p\" (UID: \"8f45205a-57e4-4819-931e-f3533ee73a92\") " pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-t6w5p"
Sep 15 20:13:04.511624 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:04.511520    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8f45205a-57e4-4819-931e-f3533ee73a92-manifests\") pod \"opendatahub-operator-controller-manager-75fcf566-t6w5p\" (UID: \"8f45205a-57e4-4819-931e-f3533ee73a92\") " pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-t6w5p"
Sep 15 20:13:04.511624 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:04.511548    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8f45205a-57e4-4819-931e-f3533ee73a92-apiservice-cert\") pod \"opendatahub-operator-controller-manager-75fcf566-t6w5p\" (UID: \"8f45205a-57e4-4819-931e-f3533ee73a92\") " pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-t6w5p"
Sep 15 20:13:04.511697 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:04.511627    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bfhb\" (UniqueName: \"kubernetes.io/projected/8f45205a-57e4-4819-931e-f3533ee73a92-kube-api-access-5bfhb\") pod \"opendatahub-operator-controller-manager-75fcf566-t6w5p\" (UID: \"8f45205a-57e4-4819-931e-f3533ee73a92\") " pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-t6w5p"
Sep 15 20:13:04.511697 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:04.511653    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f45205a-57e4-4819-931e-f3533ee73a92-webhook-cert\") pod \"opendatahub-operator-controller-manager-75fcf566-t6w5p\" (UID: \"8f45205a-57e4-4819-931e-f3533ee73a92\") " pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-t6w5p"
Sep 15 20:13:04.612833 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:04.612765    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f45205a-57e4-4819-931e-f3533ee73a92-webhook-cert\") pod \"opendatahub-operator-controller-manager-75fcf566-t6w5p\" (UID: \"8f45205a-57e4-4819-931e-f3533ee73a92\") " pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-t6w5p"
Sep 15 20:13:04.612833 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:04.612809    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f45205a-57e4-4819-931e-f3533ee73a92-tmp\") pod \"opendatahub-operator-controller-manager-75fcf566-t6w5p\" (UID: \"8f45205a-57e4-4819-931e-f3533ee73a92\") " pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-t6w5p"
Sep 15 20:13:04.612833 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:04.612830    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8f45205a-57e4-4819-931e-f3533ee73a92-manifests\") pod \"opendatahub-operator-controller-manager-75fcf566-t6w5p\" (UID: \"8f45205a-57e4-4819-931e-f3533ee73a92\") " pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-t6w5p"
Sep 15 20:13:04.613036 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:04.612858    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8f45205a-57e4-4819-931e-f3533ee73a92-apiservice-cert\") pod \"opendatahub-operator-controller-manager-75fcf566-t6w5p\" (UID: \"8f45205a-57e4-4819-931e-f3533ee73a92\") " pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-t6w5p"
Sep 15 20:13:04.613036 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:04.612896    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bfhb\" (UniqueName: \"kubernetes.io/projected/8f45205a-57e4-4819-931e-f3533ee73a92-kube-api-access-5bfhb\") pod \"opendatahub-operator-controller-manager-75fcf566-t6w5p\" (UID: \"8f45205a-57e4-4819-931e-f3533ee73a92\") " pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-t6w5p"
Sep 15 20:13:04.613321 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:04.613301    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8f45205a-57e4-4819-931e-f3533ee73a92-manifests\") pod \"opendatahub-operator-controller-manager-75fcf566-t6w5p\" (UID: \"8f45205a-57e4-4819-931e-f3533ee73a92\") " pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-t6w5p"
Sep 15 20:13:04.615403 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:04.615380    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f45205a-57e4-4819-931e-f3533ee73a92-tmp\") pod \"opendatahub-operator-controller-manager-75fcf566-t6w5p\" (UID: \"8f45205a-57e4-4819-931e-f3533ee73a92\") " pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-t6w5p"
Sep 15 20:13:04.615527 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:04.615424    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f45205a-57e4-4819-931e-f3533ee73a92-webhook-cert\") pod \"opendatahub-operator-controller-manager-75fcf566-t6w5p\" (UID: \"8f45205a-57e4-4819-931e-f3533ee73a92\") " pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-t6w5p"
Sep 15 20:13:04.615618 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:04.615597    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8f45205a-57e4-4819-931e-f3533ee73a92-apiservice-cert\") pod \"opendatahub-operator-controller-manager-75fcf566-t6w5p\" (UID: \"8f45205a-57e4-4819-931e-f3533ee73a92\") " pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-t6w5p"
Sep 15 20:13:04.625693 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:04.625666    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:04.627118 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:04.627101    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bfhb\" (UniqueName: \"kubernetes.io/projected/8f45205a-57e4-4819-931e-f3533ee73a92-kube-api-access-5bfhb\") pod \"opendatahub-operator-controller-manager-75fcf566-t6w5p\" (UID: \"8f45205a-57e4-4819-931e-f3533ee73a92\") " pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-t6w5p"
Sep 15 20:13:04.730292 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:04.730265    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:04.763718 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:04.763696    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-t6w5p"
Sep 15 20:13:04.826127 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:04.826089    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:04.926349 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:04.926278    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:05.022167 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:05.022139    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:05.042182 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:05.042161    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-75fcf566-t6w5p"]
Sep 15 20:13:05.044623 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:13:05.044602    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f45205a_57e4_4819_931e_f3533ee73a92.slice/crio-dafc1c53104c3bb0128d5d1416e2489fe50eb351df3cf5688f722aa44a04b971 WatchSource:0}: Error finding container dafc1c53104c3bb0128d5d1416e2489fe50eb351df3cf5688f722aa44a04b971: Status 404 returned error can't find the container with id dafc1c53104c3bb0128d5d1416e2489fe50eb351df3cf5688f722aa44a04b971
Sep 15 20:13:05.088864 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:05.088840    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:05.189948 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:05.189869    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:05.876871 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:05.876836    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-t6w5p" event={"ID":"8f45205a-57e4-4819-931e-f3533ee73a92","Type":"ContainerStarted","Data":"dafc1c53104c3bb0128d5d1416e2489fe50eb351df3cf5688f722aa44a04b971"}
Sep 15 20:13:08.891769 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:08.891734    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-t6w5p" event={"ID":"8f45205a-57e4-4819-931e-f3533ee73a92","Type":"ContainerStarted","Data":"d1c94ea8d04e398f666d733313b72898376529627e98352506a010da4b6197a6"}
Sep 15 20:13:09.896987 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:09.896898    2576 generic.go:356] "Generic (PLEG): container finished" podID="8f45205a-57e4-4819-931e-f3533ee73a92" containerID="d1c94ea8d04e398f666d733313b72898376529627e98352506a010da4b6197a6" exitCode=0
Sep 15 20:13:09.897311 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:09.896985    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-t6w5p" event={"ID":"8f45205a-57e4-4819-931e-f3533ee73a92","Type":"ContainerDied","Data":"d1c94ea8d04e398f666d733313b72898376529627e98352506a010da4b6197a6"}
Sep 15 20:13:11.906564 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:11.906526    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-t6w5p" event={"ID":"8f45205a-57e4-4819-931e-f3533ee73a92","Type":"ContainerStarted","Data":"de05b780fa74df7e3a72c5aada364892e1dc9d2623e2f48d3af54cd4eee6418c"}
Sep 15 20:13:11.906987 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:11.906645    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-t6w5p"
Sep 15 20:13:11.927706 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:11.927567    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-t6w5p" podStartSLOduration=2.47150809 podStartE2EDuration="7.927551227s" totalImagesPullingTime="4.419162488s" totalInitContainerRuntime="1.036880649s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:13:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:13:11.927030151 +0000 UTC m=+506.415494639" watchObservedRunningTime="2026-09-15 20:13:11.927551227 +0000 UTC m=+506.416015704"
Sep 15 20:13:12.813298 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:12.813265    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:12.915266 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:12.915235    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:14.768187 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:14.768157    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:14.860826 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:14.860795    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:16.891268 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:16.891229    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:16.982200 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:16.982169    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:22.919315 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:22.919281    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-t6w5p"
Sep 15 20:13:23.024761 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:23.024731    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-d95cdffcd-lg6sh"]
Sep 15 20:13:23.032983 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:23.032962    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d95cdffcd-lg6sh"
Sep 15 20:13:23.035651 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:23.035623    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-hcvps\""
Sep 15 20:13:23.035761 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:23.035632    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 15 20:13:23.036034 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:23.036013    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 15 20:13:23.036137 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:23.036065    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 15 20:13:23.036137 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:23.036120    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d95cdffcd-lg6sh"]
Sep 15 20:13:23.036137 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:23.036131    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 15 20:13:23.036309 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:23.036065    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 15 20:13:23.168280 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:23.168247    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a13421ec-d464-4435-8b63-76298a3becb0-manager-config\") pod \"lws-controller-manager-d95cdffcd-lg6sh\" (UID: \"a13421ec-d464-4435-8b63-76298a3becb0\") " pod="openshift-lws-operator/lws-controller-manager-d95cdffcd-lg6sh"
Sep 15 20:13:23.168441 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:23.168289    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a13421ec-d464-4435-8b63-76298a3becb0-cert\") pod \"lws-controller-manager-d95cdffcd-lg6sh\" (UID: \"a13421ec-d464-4435-8b63-76298a3becb0\") " pod="openshift-lws-operator/lws-controller-manager-d95cdffcd-lg6sh"
Sep 15 20:13:23.168441 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:23.168313    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmb6s\" (UniqueName: \"kubernetes.io/projected/a13421ec-d464-4435-8b63-76298a3becb0-kube-api-access-bmb6s\") pod \"lws-controller-manager-d95cdffcd-lg6sh\" (UID: \"a13421ec-d464-4435-8b63-76298a3becb0\") " pod="openshift-lws-operator/lws-controller-manager-d95cdffcd-lg6sh"
Sep 15 20:13:23.168441 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:23.168355    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a13421ec-d464-4435-8b63-76298a3becb0-metrics-cert\") pod \"lws-controller-manager-d95cdffcd-lg6sh\" (UID: \"a13421ec-d464-4435-8b63-76298a3becb0\") " pod="openshift-lws-operator/lws-controller-manager-d95cdffcd-lg6sh"
Sep 15 20:13:23.269623 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:23.269596    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a13421ec-d464-4435-8b63-76298a3becb0-manager-config\") pod \"lws-controller-manager-d95cdffcd-lg6sh\" (UID: \"a13421ec-d464-4435-8b63-76298a3becb0\") " pod="openshift-lws-operator/lws-controller-manager-d95cdffcd-lg6sh"
Sep 15 20:13:23.269765 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:23.269634    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a13421ec-d464-4435-8b63-76298a3becb0-cert\") pod \"lws-controller-manager-d95cdffcd-lg6sh\" (UID: \"a13421ec-d464-4435-8b63-76298a3becb0\") " pod="openshift-lws-operator/lws-controller-manager-d95cdffcd-lg6sh"
Sep 15 20:13:23.269765 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:23.269654    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmb6s\" (UniqueName: \"kubernetes.io/projected/a13421ec-d464-4435-8b63-76298a3becb0-kube-api-access-bmb6s\") pod \"lws-controller-manager-d95cdffcd-lg6sh\" (UID: \"a13421ec-d464-4435-8b63-76298a3becb0\") " pod="openshift-lws-operator/lws-controller-manager-d95cdffcd-lg6sh"
Sep 15 20:13:23.269871 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:23.269759    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a13421ec-d464-4435-8b63-76298a3becb0-metrics-cert\") pod \"lws-controller-manager-d95cdffcd-lg6sh\" (UID: \"a13421ec-d464-4435-8b63-76298a3becb0\") " pod="openshift-lws-operator/lws-controller-manager-d95cdffcd-lg6sh"
Sep 15 20:13:23.270376 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:23.270353    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a13421ec-d464-4435-8b63-76298a3becb0-manager-config\") pod \"lws-controller-manager-d95cdffcd-lg6sh\" (UID: \"a13421ec-d464-4435-8b63-76298a3becb0\") " pod="openshift-lws-operator/lws-controller-manager-d95cdffcd-lg6sh"
Sep 15 20:13:23.272056 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:23.272037    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a13421ec-d464-4435-8b63-76298a3becb0-cert\") pod \"lws-controller-manager-d95cdffcd-lg6sh\" (UID: \"a13421ec-d464-4435-8b63-76298a3becb0\") " pod="openshift-lws-operator/lws-controller-manager-d95cdffcd-lg6sh"
Sep 15 20:13:23.272517 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:23.272502    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a13421ec-d464-4435-8b63-76298a3becb0-metrics-cert\") pod \"lws-controller-manager-d95cdffcd-lg6sh\" (UID: \"a13421ec-d464-4435-8b63-76298a3becb0\") " pod="openshift-lws-operator/lws-controller-manager-d95cdffcd-lg6sh"
Sep 15 20:13:23.278523 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:23.278500    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmb6s\" (UniqueName: \"kubernetes.io/projected/a13421ec-d464-4435-8b63-76298a3becb0-kube-api-access-bmb6s\") pod \"lws-controller-manager-d95cdffcd-lg6sh\" (UID: \"a13421ec-d464-4435-8b63-76298a3becb0\") " pod="openshift-lws-operator/lws-controller-manager-d95cdffcd-lg6sh"
Sep 15 20:13:23.343461 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:23.343438    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d95cdffcd-lg6sh"
Sep 15 20:13:23.610716 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:23.610692    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d95cdffcd-lg6sh"]
Sep 15 20:13:23.613198 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:13:23.613174    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda13421ec_d464_4435_8b63_76298a3becb0.slice/crio-d5d2c3de525d3526d36a2b1c20c154cfa8a0ac9ded70b576ac67a53f98c252e7 WatchSource:0}: Error finding container d5d2c3de525d3526d36a2b1c20c154cfa8a0ac9ded70b576ac67a53f98c252e7: Status 404 returned error can't find the container with id d5d2c3de525d3526d36a2b1c20c154cfa8a0ac9ded70b576ac67a53f98c252e7
Sep 15 20:13:23.962340 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:23.962308    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d95cdffcd-lg6sh" event={"ID":"a13421ec-d464-4435-8b63-76298a3becb0","Type":"ContainerStarted","Data":"d5d2c3de525d3526d36a2b1c20c154cfa8a0ac9ded70b576ac67a53f98c252e7"}
Sep 15 20:13:24.646150 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:24.646113    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:24.783308 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:24.779482    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:26.977673 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:26.977636    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d95cdffcd-lg6sh" event={"ID":"a13421ec-d464-4435-8b63-76298a3becb0","Type":"ContainerStarted","Data":"4cf62e69a320faf8a3df487292a1f80969c8c3013f9fa9f661491cd5a7d3e6fc"}
Sep 15 20:13:26.978142 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:26.977758    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-d95cdffcd-lg6sh"
Sep 15 20:13:26.995508 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:26.995456    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-d95cdffcd-lg6sh" podStartSLOduration=1.285199562 podStartE2EDuration="3.995443195s" totalImagesPullingTime="2.710243633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:13:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:13:26.993733073 +0000 UTC m=+521.482197550" watchObservedRunningTime="2026-09-15 20:13:26.995443195 +0000 UTC m=+521.483907670"
Sep 15 20:13:32.177319 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:32.177287    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:35.371311 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:35.371248    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:35.444897 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:35.444859    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-565c49d7c-bgkjg"]
Sep 15 20:13:35.447916 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:35.447893    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-565c49d7c-bgkjg"
Sep 15 20:13:35.451683 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:35.451652    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 15 20:13:35.452082 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:35.452064    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-bztbx\""
Sep 15 20:13:35.452397 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:35.452379    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 15 20:13:35.459105 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:35.459081    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-565c49d7c-bgkjg"]
Sep 15 20:13:35.468058 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:35.468036    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzwqd\" (UniqueName: \"kubernetes.io/projected/a98edf6b-4657-4ae2-b251-596b6ac44cf2-kube-api-access-qzwqd\") pod \"kube-auth-proxy-565c49d7c-bgkjg\" (UID: \"a98edf6b-4657-4ae2-b251-596b6ac44cf2\") " pod="openshift-ingress/kube-auth-proxy-565c49d7c-bgkjg"
Sep 15 20:13:35.468159 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:35.468107    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a98edf6b-4657-4ae2-b251-596b6ac44cf2-tmp\") pod \"kube-auth-proxy-565c49d7c-bgkjg\" (UID: \"a98edf6b-4657-4ae2-b251-596b6ac44cf2\") " pod="openshift-ingress/kube-auth-proxy-565c49d7c-bgkjg"
Sep 15 20:13:35.468203 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:35.468163    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a98edf6b-4657-4ae2-b251-596b6ac44cf2-tls-certs\") pod \"kube-auth-proxy-565c49d7c-bgkjg\" (UID: \"a98edf6b-4657-4ae2-b251-596b6ac44cf2\") " pod="openshift-ingress/kube-auth-proxy-565c49d7c-bgkjg"
Sep 15 20:13:35.538061 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:35.538029    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:35.568767 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:35.568735    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a98edf6b-4657-4ae2-b251-596b6ac44cf2-tmp\") pod \"kube-auth-proxy-565c49d7c-bgkjg\" (UID: \"a98edf6b-4657-4ae2-b251-596b6ac44cf2\") " pod="openshift-ingress/kube-auth-proxy-565c49d7c-bgkjg"
Sep 15 20:13:35.568899 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:35.568774    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a98edf6b-4657-4ae2-b251-596b6ac44cf2-tls-certs\") pod \"kube-auth-proxy-565c49d7c-bgkjg\" (UID: \"a98edf6b-4657-4ae2-b251-596b6ac44cf2\") " pod="openshift-ingress/kube-auth-proxy-565c49d7c-bgkjg"
Sep 15 20:13:35.568899 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:35.568809    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzwqd\" (UniqueName: \"kubernetes.io/projected/a98edf6b-4657-4ae2-b251-596b6ac44cf2-kube-api-access-qzwqd\") pod \"kube-auth-proxy-565c49d7c-bgkjg\" (UID: \"a98edf6b-4657-4ae2-b251-596b6ac44cf2\") " pod="openshift-ingress/kube-auth-proxy-565c49d7c-bgkjg"
Sep 15 20:13:35.570980 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:35.570951    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a98edf6b-4657-4ae2-b251-596b6ac44cf2-tmp\") pod \"kube-auth-proxy-565c49d7c-bgkjg\" (UID: \"a98edf6b-4657-4ae2-b251-596b6ac44cf2\") " pod="openshift-ingress/kube-auth-proxy-565c49d7c-bgkjg"
Sep 15 20:13:35.571146 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:35.571132    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a98edf6b-4657-4ae2-b251-596b6ac44cf2-tls-certs\") pod \"kube-auth-proxy-565c49d7c-bgkjg\" (UID: \"a98edf6b-4657-4ae2-b251-596b6ac44cf2\") " pod="openshift-ingress/kube-auth-proxy-565c49d7c-bgkjg"
Sep 15 20:13:35.582997 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:35.582978    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzwqd\" (UniqueName: \"kubernetes.io/projected/a98edf6b-4657-4ae2-b251-596b6ac44cf2-kube-api-access-qzwqd\") pod \"kube-auth-proxy-565c49d7c-bgkjg\" (UID: \"a98edf6b-4657-4ae2-b251-596b6ac44cf2\") " pod="openshift-ingress/kube-auth-proxy-565c49d7c-bgkjg"
Sep 15 20:13:35.655459 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:35.655368    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:35.756493 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:35.756460    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:35.759460 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:35.759442    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-565c49d7c-bgkjg"
Sep 15 20:13:35.879082 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:35.879049    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:36.031907 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:36.031875    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-565c49d7c-bgkjg"]
Sep 15 20:13:36.035828 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:13:36.035790    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda98edf6b_4657_4ae2_b251_596b6ac44cf2.slice/crio-e9a0aebd73904f524cf6bd7f413767f69f244062ea7edcf3ab3b09bc56c6ec0a WatchSource:0}: Error finding container e9a0aebd73904f524cf6bd7f413767f69f244062ea7edcf3ab3b09bc56c6ec0a: Status 404 returned error can't find the container with id e9a0aebd73904f524cf6bd7f413767f69f244062ea7edcf3ab3b09bc56c6ec0a
Sep 15 20:13:36.232833 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:36.232796    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:36.615497 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:36.615451    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:36.897764 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:36.897658    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:36.971206 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:36.971175    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:37.023126 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:37.023079    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-565c49d7c-bgkjg" event={"ID":"a98edf6b-4657-4ae2-b251-596b6ac44cf2","Type":"ContainerStarted","Data":"e9a0aebd73904f524cf6bd7f413767f69f244062ea7edcf3ab3b09bc56c6ec0a"}
Sep 15 20:13:37.862996 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:37.862949    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:37.985015 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:37.984983    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-d95cdffcd-lg6sh"
Sep 15 20:13:39.034390 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:39.034358    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-565c49d7c-bgkjg" event={"ID":"a98edf6b-4657-4ae2-b251-596b6ac44cf2","Type":"ContainerStarted","Data":"d73ffe4373626630d4cdbd18fd4a8e4248783b93fbe7645cce3ed969cc4ae816"}
Sep 15 20:13:39.052693 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:39.052635    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-565c49d7c-bgkjg" podStartSLOduration=1.128620198 podStartE2EDuration="4.052618995s" totalImagesPullingTime="2.923998797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:13:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:13:39.052173037 +0000 UTC m=+533.540637514" watchObservedRunningTime="2026-09-15 20:13:39.052618995 +0000 UTC m=+533.541083470"
Sep 15 20:13:43.759688 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:43.759651    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:43.860151 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:43.860112    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:44.167403 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:44.167326    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:44.396298 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:44.396268    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:44.495482 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:44.495449    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:45.859028 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:45.858988    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:45.954964 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:45.954933    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:46.057034 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:46.056998    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:46.335418 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:46.335384    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:47.281034 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:47.280993    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:47.396136 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:47.396106    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:47.485976 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:47.485948    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:52.902696 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:52.902656    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:52.967213 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:52.967181    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:55.099975 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:55.099938    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:13:55.153337 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:13:55.153302    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:14:36.751375 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:14:36.751343    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:14:38.412411 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:14:38.412354    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:14:48.709915 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:14:48.709885    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:14:48.767529 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:14:48.767501    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:14:48.824753 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:14:48.824723    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:14:51.234037 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:14:51.234009    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:14:52.025694 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:14:52.025664    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:14:55.094934 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:14:55.094890    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:14:55.278738 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:14:55.278700    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:14:56.494851 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:14:56.494823    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:14:57.437012 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:14:57.436981    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:14:57.830194 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:14:57.830160    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:14:58.088341 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:14:58.088265    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:14:58.191595 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:14:58.191564    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:14:58.298604 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:14:58.298577    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:15:01.683544 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:01.683501    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:15:02.729648 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:02.729617    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:15:03.142543 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:03.142465    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-x4sct"]
Sep 15 20:15:03.145758 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:03.145742    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-x4sct"
Sep 15 20:15:03.148455 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:03.148433    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 15 20:15:03.148607 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:03.148590    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 15 20:15:03.149244 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:03.149230    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-x7nhb\""
Sep 15 20:15:03.164876 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:03.164856    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-x4sct"]
Sep 15 20:15:03.284075 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:03.284039    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nckwg\" (UniqueName: \"kubernetes.io/projected/4b87edf6-1145-45ed-9146-2c013f820476-kube-api-access-nckwg\") pod \"limitador-operator-controller-manager-74495c69f8-x4sct\" (UID: \"4b87edf6-1145-45ed-9146-2c013f820476\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-x4sct"
Sep 15 20:15:03.384692 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:03.384655    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nckwg\" (UniqueName: \"kubernetes.io/projected/4b87edf6-1145-45ed-9146-2c013f820476-kube-api-access-nckwg\") pod \"limitador-operator-controller-manager-74495c69f8-x4sct\" (UID: \"4b87edf6-1145-45ed-9146-2c013f820476\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-x4sct"
Sep 15 20:15:03.392604 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:03.392543    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nckwg\" (UniqueName: \"kubernetes.io/projected/4b87edf6-1145-45ed-9146-2c013f820476-kube-api-access-nckwg\") pod \"limitador-operator-controller-manager-74495c69f8-x4sct\" (UID: \"4b87edf6-1145-45ed-9146-2c013f820476\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-x4sct"
Sep 15 20:15:03.455333 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:03.455302    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-x4sct"
Sep 15 20:15:03.728429 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:03.728395    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-x4sct"]
Sep 15 20:15:03.733302 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:15:03.733273    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b87edf6_1145_45ed_9146_2c013f820476.slice/crio-f3ffdba31bfe308546db3097439d4ca3a63a413cd81239e5bfab29e382a43bee WatchSource:0}: Error finding container f3ffdba31bfe308546db3097439d4ca3a63a413cd81239e5bfab29e382a43bee: Status 404 returned error can't find the container with id f3ffdba31bfe308546db3097439d4ca3a63a413cd81239e5bfab29e382a43bee
Sep 15 20:15:03.735389 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:03.735371    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 20:15:04.400733 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:04.400701    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-x4sct" event={"ID":"4b87edf6-1145-45ed-9146-2c013f820476","Type":"ContainerStarted","Data":"f3ffdba31bfe308546db3097439d4ca3a63a413cd81239e5bfab29e382a43bee"}
Sep 15 20:15:04.652827 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:04.652731    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:15:04.762722 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:04.762616    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:15:05.040994 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:05.040936    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:15:05.101946 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:05.101894    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:15:06.411895 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:06.411867    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-x4sct" event={"ID":"4b87edf6-1145-45ed-9146-2c013f820476","Type":"ContainerStarted","Data":"3e72caa2c9fcd81b049ad1f409c64511da576f61e21ea6987e249e4d6170d343"}
Sep 15 20:15:06.412204 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:06.411991    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-x4sct"
Sep 15 20:15:06.434939 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:06.434877    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-x4sct" podStartSLOduration=0.857606808 podStartE2EDuration="3.434860783s" totalImagesPullingTime="2.577253975s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:15:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:15:06.432286401 +0000 UTC m=+620.920750877" watchObservedRunningTime="2026-09-15 20:15:06.434860783 +0000 UTC m=+620.923325259"
Sep 15 20:15:08.257794 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:08.257712    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:15:08.328360 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:08.328326    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:15:09.093771 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:09.093735    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:15:09.413755 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:09.413669    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:15:13.318874 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:13.318823    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:15:13.416985 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:13.416955    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:15:13.518232 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:13.518205    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:15:17.418785 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:17.418757    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-x4sct"
Sep 15 20:15:19.961301 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:19.961268    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-x4sct"]
Sep 15 20:15:19.961749 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:19.961501    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-x4sct" podUID="4b87edf6-1145-45ed-9146-2c013f820476" containerName="manager" containerID="cri-o://3e72caa2c9fcd81b049ad1f409c64511da576f61e21ea6987e249e4d6170d343" gracePeriod=2
Sep 15 20:15:19.973227 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:19.973183    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-x4sct"]
Sep 15 20:15:19.989109 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:19.989087    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9k8tc"]
Sep 15 20:15:19.989425 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:19.989413    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b87edf6-1145-45ed-9146-2c013f820476" containerName="manager"
Sep 15 20:15:19.989469 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:19.989427    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b87edf6-1145-45ed-9146-2c013f820476" containerName="manager"
Sep 15 20:15:19.989506 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:19.989491    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b87edf6-1145-45ed-9146-2c013f820476" containerName="manager"
Sep 15 20:15:19.992533 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:19.992512    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9k8tc"
Sep 15 20:15:19.993333 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:19.993309    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfzbk"]
Sep 15 20:15:19.994432 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:15:19.994407    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-x4sct\" is forbidden: User \"system:node:ip-10-0-134-155.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-155.ec2.internal' and this object" podUID="4b87edf6-1145-45ed-9146-2c013f820476" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-x4sct"
Sep 15 20:15:19.994804 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:19.994787    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dbq9q\""
Sep 15 20:15:19.996533 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:19.996519    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfzbk"
Sep 15 20:15:20.013134 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:20.013107    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9k8tc"]
Sep 15 20:15:20.014488 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:20.014465    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfzbk"]
Sep 15 20:15:20.023195 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:20.023175    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j4pg\" (UniqueName: \"kubernetes.io/projected/4f54ed9c-dbe4-42c6-86cf-caa1671c425d-kube-api-access-9j4pg\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9k8tc\" (UID: \"4f54ed9c-dbe4-42c6-86cf-caa1671c425d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9k8tc"
Sep 15 20:15:20.023320 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:20.023207    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f54ed9c-dbe4-42c6-86cf-caa1671c425d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9k8tc\" (UID: \"4f54ed9c-dbe4-42c6-86cf-caa1671c425d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9k8tc"
Sep 15 20:15:20.023320 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:20.023230    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5xnv\" (UniqueName: \"kubernetes.io/projected/0ed6bbae-de4f-41ce-ae10-0cd3175cb10e-kube-api-access-x5xnv\") pod \"limitador-operator-controller-manager-74495c69f8-dfzbk\" (UID: \"0ed6bbae-de4f-41ce-ae10-0cd3175cb10e\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfzbk"
Sep 15 20:15:20.044849 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:15:20.044821    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-x4sct\" is forbidden: User \"system:node:ip-10-0-134-155.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-155.ec2.internal' and this object" podUID="4b87edf6-1145-45ed-9146-2c013f820476" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-x4sct"
Sep 15 20:15:20.124199 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:20.124154    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j4pg\" (UniqueName: \"kubernetes.io/projected/4f54ed9c-dbe4-42c6-86cf-caa1671c425d-kube-api-access-9j4pg\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9k8tc\" (UID: \"4f54ed9c-dbe4-42c6-86cf-caa1671c425d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9k8tc"
Sep 15 20:15:20.124375 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:20.124208    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f54ed9c-dbe4-42c6-86cf-caa1671c425d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9k8tc\" (UID: \"4f54ed9c-dbe4-42c6-86cf-caa1671c425d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9k8tc"
Sep 15 20:15:20.124375 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:20.124243    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5xnv\" (UniqueName: \"kubernetes.io/projected/0ed6bbae-de4f-41ce-ae10-0cd3175cb10e-kube-api-access-x5xnv\") pod \"limitador-operator-controller-manager-74495c69f8-dfzbk\" (UID: \"0ed6bbae-de4f-41ce-ae10-0cd3175cb10e\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfzbk"
Sep 15 20:15:20.124672 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:20.124648    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f54ed9c-dbe4-42c6-86cf-caa1671c425d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9k8tc\" (UID: \"4f54ed9c-dbe4-42c6-86cf-caa1671c425d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9k8tc"
Sep 15 20:15:20.141760 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:20.141736    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5xnv\" (UniqueName: \"kubernetes.io/projected/0ed6bbae-de4f-41ce-ae10-0cd3175cb10e-kube-api-access-x5xnv\") pod \"limitador-operator-controller-manager-74495c69f8-dfzbk\" (UID: \"0ed6bbae-de4f-41ce-ae10-0cd3175cb10e\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfzbk"
Sep 15 20:15:20.142182 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:20.142162    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j4pg\" (UniqueName: \"kubernetes.io/projected/4f54ed9c-dbe4-42c6-86cf-caa1671c425d-kube-api-access-9j4pg\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9k8tc\" (UID: \"4f54ed9c-dbe4-42c6-86cf-caa1671c425d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9k8tc"
Sep 15 20:15:20.178087 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:20.178062    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:15:20.195307 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:20.195289    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-x4sct"
Sep 15 20:15:20.326501 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:20.326415    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nckwg\" (UniqueName: \"kubernetes.io/projected/4b87edf6-1145-45ed-9146-2c013f820476-kube-api-access-nckwg\") pod \"4b87edf6-1145-45ed-9146-2c013f820476\" (UID: \"4b87edf6-1145-45ed-9146-2c013f820476\") "
Sep 15 20:15:20.328817 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:20.328792    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b87edf6-1145-45ed-9146-2c013f820476-kube-api-access-nckwg" (OuterVolumeSpecName: "kube-api-access-nckwg") pod "4b87edf6-1145-45ed-9146-2c013f820476" (UID: "4b87edf6-1145-45ed-9146-2c013f820476"). InnerVolumeSpecName "kube-api-access-nckwg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:15:20.338957 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:20.338916    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9k8tc"
Sep 15 20:15:20.345640 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:20.345622    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfzbk"
Sep 15 20:15:20.386276 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:20.386213    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:15:20.427301 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:20.427270    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nckwg\" (UniqueName: \"kubernetes.io/projected/4b87edf6-1145-45ed-9146-2c013f820476-kube-api-access-nckwg\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:15:20.473085 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:20.473056    2576 generic.go:356] "Generic (PLEG): container finished" podID="4b87edf6-1145-45ed-9146-2c013f820476" containerID="3e72caa2c9fcd81b049ad1f409c64511da576f61e21ea6987e249e4d6170d343" exitCode=0
Sep 15 20:15:20.473188 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:20.473100    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-x4sct"
Sep 15 20:15:20.473188 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:20.473124    2576 scope.go:117] "RemoveContainer" containerID="3e72caa2c9fcd81b049ad1f409c64511da576f61e21ea6987e249e4d6170d343"
Sep 15 20:15:20.481193 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:20.481175    2576 scope.go:117] "RemoveContainer" containerID="3e72caa2c9fcd81b049ad1f409c64511da576f61e21ea6987e249e4d6170d343"
Sep 15 20:15:20.481440 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:15:20.481416    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e72caa2c9fcd81b049ad1f409c64511da576f61e21ea6987e249e4d6170d343\": container with ID starting with 3e72caa2c9fcd81b049ad1f409c64511da576f61e21ea6987e249e4d6170d343 not found: ID does not exist" containerID="3e72caa2c9fcd81b049ad1f409c64511da576f61e21ea6987e249e4d6170d343"
Sep 15 20:15:20.481495 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:20.481443    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e72caa2c9fcd81b049ad1f409c64511da576f61e21ea6987e249e4d6170d343"} err="failed to get container status \"3e72caa2c9fcd81b049ad1f409c64511da576f61e21ea6987e249e4d6170d343\": rpc error: code = NotFound desc = could not find container \"3e72caa2c9fcd81b049ad1f409c64511da576f61e21ea6987e249e4d6170d343\": container with ID starting with 3e72caa2c9fcd81b049ad1f409c64511da576f61e21ea6987e249e4d6170d343 not found: ID does not exist"
Sep 15 20:15:20.487846 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:20.487826    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:15:20.616639 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:20.616617    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9k8tc"]
Sep 15 20:15:20.618671 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:15:20.618643    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f54ed9c_dbe4_42c6_86cf_caa1671c425d.slice/crio-76dce4546459177e9ab36ffee340ae482f9b558b8d3339705442aad42548d683 WatchSource:0}: Error finding container 76dce4546459177e9ab36ffee340ae482f9b558b8d3339705442aad42548d683: Status 404 returned error can't find the container with id 76dce4546459177e9ab36ffee340ae482f9b558b8d3339705442aad42548d683
Sep 15 20:15:20.623534 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:20.623429    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfzbk"]
Sep 15 20:15:20.626171 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:15:20.626148    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ed6bbae_de4f_41ce_ae10_0cd3175cb10e.slice/crio-856fd307d4aada5dd522614f8669d68a6cf791fff5364dd3ccf6f2662d2c07dd WatchSource:0}: Error finding container 856fd307d4aada5dd522614f8669d68a6cf791fff5364dd3ccf6f2662d2c07dd: Status 404 returned error can't find the container with id 856fd307d4aada5dd522614f8669d68a6cf791fff5364dd3ccf6f2662d2c07dd
Sep 15 20:15:21.481126 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:21.481087    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9k8tc" event={"ID":"4f54ed9c-dbe4-42c6-86cf-caa1671c425d","Type":"ContainerStarted","Data":"76dce4546459177e9ab36ffee340ae482f9b558b8d3339705442aad42548d683"}
Sep 15 20:15:21.483933 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:21.483899    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfzbk" event={"ID":"0ed6bbae-de4f-41ce-ae10-0cd3175cb10e","Type":"ContainerStarted","Data":"298a1c007baff970f087dfe3cbd4091b6c789374edf967e0ddb69ddc6aac8ec9"}
Sep 15 20:15:21.484061 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:21.483964    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfzbk" event={"ID":"0ed6bbae-de4f-41ce-ae10-0cd3175cb10e","Type":"ContainerStarted","Data":"856fd307d4aada5dd522614f8669d68a6cf791fff5364dd3ccf6f2662d2c07dd"}
Sep 15 20:15:21.484132 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:21.484074    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfzbk"
Sep 15 20:15:21.508549 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:21.508497    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfzbk" podStartSLOduration=2.508482468 podStartE2EDuration="2.508482468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:15:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:15:21.50845908 +0000 UTC m=+635.996923557" watchObservedRunningTime="2026-09-15 20:15:21.508482468 +0000 UTC m=+635.996946946"
Sep 15 20:15:22.145166 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:22.145131    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b87edf6-1145-45ed-9146-2c013f820476" path="/var/lib/kubelet/pods/4b87edf6-1145-45ed-9146-2c013f820476/volumes"
Sep 15 20:15:24.165579 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:24.165497    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:15:24.260706 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:24.260673    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:15:24.500248 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:24.500214    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9k8tc" event={"ID":"4f54ed9c-dbe4-42c6-86cf-caa1671c425d","Type":"ContainerStarted","Data":"97c4f4b21fecc6256106f54ea4901b32510309cf7e4d684e6dbc9dabac3b8dad"}
Sep 15 20:15:24.500390 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:24.500342    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9k8tc"
Sep 15 20:15:24.527410 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:24.527367    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9k8tc" podStartSLOduration=2.309178266 podStartE2EDuration="5.52735252s" totalImagesPullingTime="3.218174254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:15:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:15:24.524836331 +0000 UTC m=+639.013300808" watchObservedRunningTime="2026-09-15 20:15:24.52735252 +0000 UTC m=+639.015816995"
Sep 15 20:15:24.751005 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:24.750913    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:15:24.850563 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:24.850531    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:15:27.147613 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:27.147581    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:15:27.248426 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:27.248399    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:15:31.853238 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:31.853203    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9k8tc"]
Sep 15 20:15:31.853629 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:31.853412    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9k8tc" podUID="4f54ed9c-dbe4-42c6-86cf-caa1671c425d" containerName="manager" containerID="cri-o://97c4f4b21fecc6256106f54ea4901b32510309cf7e4d684e6dbc9dabac3b8dad" gracePeriod=10
Sep 15 20:15:31.855108 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:31.855084    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9k8tc"
Sep 15 20:15:32.092347 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:32.092327    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9k8tc"
Sep 15 20:15:32.121466 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:32.121396    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9j4pg\" (UniqueName: \"kubernetes.io/projected/4f54ed9c-dbe4-42c6-86cf-caa1671c425d-kube-api-access-9j4pg\") pod \"4f54ed9c-dbe4-42c6-86cf-caa1671c425d\" (UID: \"4f54ed9c-dbe4-42c6-86cf-caa1671c425d\") "
Sep 15 20:15:32.121466 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:32.121450    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f54ed9c-dbe4-42c6-86cf-caa1671c425d-extensions-socket-volume\") pod \"4f54ed9c-dbe4-42c6-86cf-caa1671c425d\" (UID: \"4f54ed9c-dbe4-42c6-86cf-caa1671c425d\") "
Sep 15 20:15:32.122069 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:32.122036    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f54ed9c-dbe4-42c6-86cf-caa1671c425d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4f54ed9c-dbe4-42c6-86cf-caa1671c425d" (UID: "4f54ed9c-dbe4-42c6-86cf-caa1671c425d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:15:32.123539 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:32.123504    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f54ed9c-dbe4-42c6-86cf-caa1671c425d-kube-api-access-9j4pg" (OuterVolumeSpecName: "kube-api-access-9j4pg") pod "4f54ed9c-dbe4-42c6-86cf-caa1671c425d" (UID: "4f54ed9c-dbe4-42c6-86cf-caa1671c425d"). InnerVolumeSpecName "kube-api-access-9j4pg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:15:32.222281 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:32.222251    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9j4pg\" (UniqueName: \"kubernetes.io/projected/4f54ed9c-dbe4-42c6-86cf-caa1671c425d-kube-api-access-9j4pg\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:15:32.222281 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:32.222279    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f54ed9c-dbe4-42c6-86cf-caa1671c425d-extensions-socket-volume\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:15:32.492053 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:32.492028    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfzbk"
Sep 15 20:15:32.536776 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:32.536745    2576 generic.go:356] "Generic (PLEG): container finished" podID="4f54ed9c-dbe4-42c6-86cf-caa1671c425d" containerID="97c4f4b21fecc6256106f54ea4901b32510309cf7e4d684e6dbc9dabac3b8dad" exitCode=0
Sep 15 20:15:32.536947 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:32.536786    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9k8tc"
Sep 15 20:15:32.536947 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:32.536824    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9k8tc" event={"ID":"4f54ed9c-dbe4-42c6-86cf-caa1671c425d","Type":"ContainerDied","Data":"97c4f4b21fecc6256106f54ea4901b32510309cf7e4d684e6dbc9dabac3b8dad"}
Sep 15 20:15:32.536947 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:32.536874    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9k8tc" event={"ID":"4f54ed9c-dbe4-42c6-86cf-caa1671c425d","Type":"ContainerDied","Data":"76dce4546459177e9ab36ffee340ae482f9b558b8d3339705442aad42548d683"}
Sep 15 20:15:32.536947 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:32.536899    2576 scope.go:117] "RemoveContainer" containerID="97c4f4b21fecc6256106f54ea4901b32510309cf7e4d684e6dbc9dabac3b8dad"
Sep 15 20:15:32.548622 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:32.548604    2576 scope.go:117] "RemoveContainer" containerID="97c4f4b21fecc6256106f54ea4901b32510309cf7e4d684e6dbc9dabac3b8dad"
Sep 15 20:15:32.548931 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:15:32.548899    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97c4f4b21fecc6256106f54ea4901b32510309cf7e4d684e6dbc9dabac3b8dad\": container with ID starting with 97c4f4b21fecc6256106f54ea4901b32510309cf7e4d684e6dbc9dabac3b8dad not found: ID does not exist" containerID="97c4f4b21fecc6256106f54ea4901b32510309cf7e4d684e6dbc9dabac3b8dad"
Sep 15 20:15:32.549014 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:32.548946    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97c4f4b21fecc6256106f54ea4901b32510309cf7e4d684e6dbc9dabac3b8dad"} err="failed to get container status \"97c4f4b21fecc6256106f54ea4901b32510309cf7e4d684e6dbc9dabac3b8dad\": rpc error: code = NotFound desc = could not find container \"97c4f4b21fecc6256106f54ea4901b32510309cf7e4d684e6dbc9dabac3b8dad\": container with ID starting with 97c4f4b21fecc6256106f54ea4901b32510309cf7e4d684e6dbc9dabac3b8dad not found: ID does not exist"
Sep 15 20:15:32.559299 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:32.559277    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9k8tc"]
Sep 15 20:15:32.563141 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:32.563121    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9k8tc"]
Sep 15 20:15:33.135224 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:33.135197    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:15:33.252240 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:33.252213    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:15:33.334854 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:33.334824    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:15:34.142390 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:34.142359    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f54ed9c-dbe4-42c6-86cf-caa1671c425d" path="/var/lib/kubelet/pods/4f54ed9c-dbe4-42c6-86cf-caa1671c425d/volumes"
Sep 15 20:15:40.159800 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:40.159752    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:15:40.259309 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:40.259275    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:15:48.300806 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:48.300773    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:15:48.523512 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:15:48.523481    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:16:01.726534 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:01.726499    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:16:01.833208 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:01.833175    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:16:01.932868 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:01.932834    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:16:02.520286 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:02.520251    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-lj5k5"]
Sep 15 20:16:02.520731 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:02.520713    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f54ed9c-dbe4-42c6-86cf-caa1671c425d" containerName="manager"
Sep 15 20:16:02.520731 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:02.520732    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f54ed9c-dbe4-42c6-86cf-caa1671c425d" containerName="manager"
Sep 15 20:16:02.520856 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:02.520834    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f54ed9c-dbe4-42c6-86cf-caa1671c425d" containerName="manager"
Sep 15 20:16:02.523776 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:02.523757    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-lj5k5"
Sep 15 20:16:02.525910 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:02.525879    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-t75c5\""
Sep 15 20:16:02.532813 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:02.532794    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-lj5k5"]
Sep 15 20:16:02.552503 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:02.552477    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:16:02.626467 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:02.626436    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:16:02.630060 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:02.630035    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-cx2j5"]
Sep 15 20:16:02.637608 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:02.637587    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-cx2j5"
Sep 15 20:16:02.642438 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:02.642417    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-cx2j5"]
Sep 15 20:16:02.670222 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:02.670199    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6fh2\" (UniqueName: \"kubernetes.io/projected/625c14d4-0b2a-4e1b-819c-e5408a1c384d-kube-api-access-h6fh2\") pod \"authorino-7bcfd8f9cd-lj5k5\" (UID: \"625c14d4-0b2a-4e1b-819c-e5408a1c384d\") " pod="kuadrant-system/authorino-7bcfd8f9cd-lj5k5"
Sep 15 20:16:02.771021 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:02.770944    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zmpl\" (UniqueName: \"kubernetes.io/projected/3c314d5f-1209-4674-ada6-dd67e4152142-kube-api-access-8zmpl\") pod \"authorino-59c974bd48-cx2j5\" (UID: \"3c314d5f-1209-4674-ada6-dd67e4152142\") " pod="kuadrant-system/authorino-59c974bd48-cx2j5"
Sep 15 20:16:02.771021 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:02.770982    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6fh2\" (UniqueName: \"kubernetes.io/projected/625c14d4-0b2a-4e1b-819c-e5408a1c384d-kube-api-access-h6fh2\") pod \"authorino-7bcfd8f9cd-lj5k5\" (UID: \"625c14d4-0b2a-4e1b-819c-e5408a1c384d\") " pod="kuadrant-system/authorino-7bcfd8f9cd-lj5k5"
Sep 15 20:16:02.779626 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:02.779577    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6fh2\" (UniqueName: \"kubernetes.io/projected/625c14d4-0b2a-4e1b-819c-e5408a1c384d-kube-api-access-h6fh2\") pod \"authorino-7bcfd8f9cd-lj5k5\" (UID: \"625c14d4-0b2a-4e1b-819c-e5408a1c384d\") " pod="kuadrant-system/authorino-7bcfd8f9cd-lj5k5"
Sep 15 20:16:02.842283 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:02.842247    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-lj5k5"
Sep 15 20:16:02.872459 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:02.872431    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zmpl\" (UniqueName: \"kubernetes.io/projected/3c314d5f-1209-4674-ada6-dd67e4152142-kube-api-access-8zmpl\") pod \"authorino-59c974bd48-cx2j5\" (UID: \"3c314d5f-1209-4674-ada6-dd67e4152142\") " pod="kuadrant-system/authorino-59c974bd48-cx2j5"
Sep 15 20:16:02.880949 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:02.880892    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zmpl\" (UniqueName: \"kubernetes.io/projected/3c314d5f-1209-4674-ada6-dd67e4152142-kube-api-access-8zmpl\") pod \"authorino-59c974bd48-cx2j5\" (UID: \"3c314d5f-1209-4674-ada6-dd67e4152142\") " pod="kuadrant-system/authorino-59c974bd48-cx2j5"
Sep 15 20:16:02.948608 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:02.948580    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-cx2j5"
Sep 15 20:16:03.113914 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:03.113829    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-lj5k5"]
Sep 15 20:16:03.116544 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:16:03.116515    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod625c14d4_0b2a_4e1b_819c_e5408a1c384d.slice/crio-a7bffb87dfad898339c48b9c90b11f16c0b57d9fb3262cc0460fba4e39d5d689 WatchSource:0}: Error finding container a7bffb87dfad898339c48b9c90b11f16c0b57d9fb3262cc0460fba4e39d5d689: Status 404 returned error can't find the container with id a7bffb87dfad898339c48b9c90b11f16c0b57d9fb3262cc0460fba4e39d5d689
Sep 15 20:16:03.215563 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:03.215542    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-cx2j5"]
Sep 15 20:16:03.217955 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:16:03.217917    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3c314d5f_1209_4674_ada6_dd67e4152142.slice/crio-4f0d27038d22ba944cc32d934408fba62f25a59f6d8e06d6bfe8ff87991c07d9 WatchSource:0}: Error finding container 4f0d27038d22ba944cc32d934408fba62f25a59f6d8e06d6bfe8ff87991c07d9: Status 404 returned error can't find the container with id 4f0d27038d22ba944cc32d934408fba62f25a59f6d8e06d6bfe8ff87991c07d9
Sep 15 20:16:03.677666 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:03.677630    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-lj5k5" event={"ID":"625c14d4-0b2a-4e1b-819c-e5408a1c384d","Type":"ContainerStarted","Data":"a7bffb87dfad898339c48b9c90b11f16c0b57d9fb3262cc0460fba4e39d5d689"}
Sep 15 20:16:03.678714 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:03.678691    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-cx2j5" event={"ID":"3c314d5f-1209-4674-ada6-dd67e4152142","Type":"ContainerStarted","Data":"4f0d27038d22ba944cc32d934408fba62f25a59f6d8e06d6bfe8ff87991c07d9"}
Sep 15 20:16:05.690402 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:05.690358    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-cx2j5" event={"ID":"3c314d5f-1209-4674-ada6-dd67e4152142","Type":"ContainerStarted","Data":"b4d52d6874a9e5c3f24baa38037d4c02f73ae9f61c142716ac59e09a5e10e6fd"}
Sep 15 20:16:05.705885 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:05.705838    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-cx2j5" podStartSLOduration=1.922955096 podStartE2EDuration="3.705823331s" totalImagesPullingTime="1.782868235s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:16:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:16:05.704657991 +0000 UTC m=+680.193122469" watchObservedRunningTime="2026-09-15 20:16:05.705823331 +0000 UTC m=+680.194287814"
Sep 15 20:16:05.730004 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:05.729977    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-lj5k5"]
Sep 15 20:16:07.700836 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:07.700802    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-lj5k5" event={"ID":"625c14d4-0b2a-4e1b-819c-e5408a1c384d","Type":"ContainerStarted","Data":"ce4863c4fbfb99cedc7b5fb1dd40f6c4c71b51f320d8eadc606673380c691738"}
Sep 15 20:16:07.701268 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:07.700910    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-lj5k5" podUID="625c14d4-0b2a-4e1b-819c-e5408a1c384d" containerName="authorino" containerID="cri-o://ce4863c4fbfb99cedc7b5fb1dd40f6c4c71b51f320d8eadc606673380c691738" gracePeriod=30
Sep 15 20:16:07.940678 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:07.940652    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-lj5k5"
Sep 15 20:16:08.119717 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:08.119686    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h6fh2\" (UniqueName: \"kubernetes.io/projected/625c14d4-0b2a-4e1b-819c-e5408a1c384d-kube-api-access-h6fh2\") pod \"625c14d4-0b2a-4e1b-819c-e5408a1c384d\" (UID: \"625c14d4-0b2a-4e1b-819c-e5408a1c384d\") "
Sep 15 20:16:08.121705 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:08.121669    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/625c14d4-0b2a-4e1b-819c-e5408a1c384d-kube-api-access-h6fh2" (OuterVolumeSpecName: "kube-api-access-h6fh2") pod "625c14d4-0b2a-4e1b-819c-e5408a1c384d" (UID: "625c14d4-0b2a-4e1b-819c-e5408a1c384d"). InnerVolumeSpecName "kube-api-access-h6fh2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:16:08.220363 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:08.220337    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6fh2\" (UniqueName: \"kubernetes.io/projected/625c14d4-0b2a-4e1b-819c-e5408a1c384d-kube-api-access-h6fh2\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:16:08.706260 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:08.706224    2576 generic.go:356] "Generic (PLEG): container finished" podID="625c14d4-0b2a-4e1b-819c-e5408a1c384d" containerID="ce4863c4fbfb99cedc7b5fb1dd40f6c4c71b51f320d8eadc606673380c691738" exitCode=0
Sep 15 20:16:08.706260 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:08.706264    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-lj5k5"
Sep 15 20:16:08.706760 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:08.706281    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-lj5k5" event={"ID":"625c14d4-0b2a-4e1b-819c-e5408a1c384d","Type":"ContainerDied","Data":"ce4863c4fbfb99cedc7b5fb1dd40f6c4c71b51f320d8eadc606673380c691738"}
Sep 15 20:16:08.706760 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:08.706310    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-lj5k5" event={"ID":"625c14d4-0b2a-4e1b-819c-e5408a1c384d","Type":"ContainerDied","Data":"a7bffb87dfad898339c48b9c90b11f16c0b57d9fb3262cc0460fba4e39d5d689"}
Sep 15 20:16:08.706760 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:08.706326    2576 scope.go:117] "RemoveContainer" containerID="ce4863c4fbfb99cedc7b5fb1dd40f6c4c71b51f320d8eadc606673380c691738"
Sep 15 20:16:08.714417 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:08.714399    2576 scope.go:117] "RemoveContainer" containerID="ce4863c4fbfb99cedc7b5fb1dd40f6c4c71b51f320d8eadc606673380c691738"
Sep 15 20:16:08.714651 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:16:08.714632    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce4863c4fbfb99cedc7b5fb1dd40f6c4c71b51f320d8eadc606673380c691738\": container with ID starting with ce4863c4fbfb99cedc7b5fb1dd40f6c4c71b51f320d8eadc606673380c691738 not found: ID does not exist" containerID="ce4863c4fbfb99cedc7b5fb1dd40f6c4c71b51f320d8eadc606673380c691738"
Sep 15 20:16:08.714708 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:08.714658    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce4863c4fbfb99cedc7b5fb1dd40f6c4c71b51f320d8eadc606673380c691738"} err="failed to get container status \"ce4863c4fbfb99cedc7b5fb1dd40f6c4c71b51f320d8eadc606673380c691738\": rpc error: code = NotFound desc = could not find container \"ce4863c4fbfb99cedc7b5fb1dd40f6c4c71b51f320d8eadc606673380c691738\": container with ID starting with ce4863c4fbfb99cedc7b5fb1dd40f6c4c71b51f320d8eadc606673380c691738 not found: ID does not exist"
Sep 15 20:16:08.722453 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:08.722428    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-lj5k5"]
Sep 15 20:16:08.726315 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:08.726295    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-lj5k5"]
Sep 15 20:16:10.142754 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:10.142720    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="625c14d4-0b2a-4e1b-819c-e5408a1c384d" path="/var/lib/kubelet/pods/625c14d4-0b2a-4e1b-819c-e5408a1c384d/volumes"
Sep 15 20:16:19.928850 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:19.928813    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:16:20.026856 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:20.026823    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:16:20.441090 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:20.441056    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:16:30.543841 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:30.543805    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:16:30.755365 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:30.755332    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:16:30.816170 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:30.816101    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-6h7x7"]
Sep 15 20:16:30.816460 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:30.816447    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="625c14d4-0b2a-4e1b-819c-e5408a1c384d" containerName="authorino"
Sep 15 20:16:30.816506 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:30.816461    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="625c14d4-0b2a-4e1b-819c-e5408a1c384d" containerName="authorino"
Sep 15 20:16:30.816540 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:30.816514    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="625c14d4-0b2a-4e1b-819c-e5408a1c384d" containerName="authorino"
Sep 15 20:16:30.819439 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:30.819424    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-6h7x7"
Sep 15 20:16:30.831233 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:30.831209    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-6h7x7"]
Sep 15 20:16:30.859537 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:30.859515    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-6h7x7"]
Sep 15 20:16:30.859810 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:16:30.859792    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-48k4p], unattached volumes=[], failed to process volumes=[kube-api-access-48k4p]: context canceled" pod="kuadrant-system/authorino-557f876f69-6h7x7" podUID="8781f715-6477-43e5-b369-fa73fe153c96"
Sep 15 20:16:30.890433 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:30.890410    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-n4wx2"]
Sep 15 20:16:30.893657 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:30.893642    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-n4wx2"
Sep 15 20:16:30.895938 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:30.895909    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 15 20:16:30.901659 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:30.901635    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-n4wx2"]
Sep 15 20:16:30.908557 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:30.908537    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48k4p\" (UniqueName: \"kubernetes.io/projected/8781f715-6477-43e5-b369-fa73fe153c96-kube-api-access-48k4p\") pod \"authorino-557f876f69-6h7x7\" (UID: \"8781f715-6477-43e5-b369-fa73fe153c96\") " pod="kuadrant-system/authorino-557f876f69-6h7x7"
Sep 15 20:16:31.009831 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:31.009799    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9wnn\" (UniqueName: \"kubernetes.io/projected/141013db-d6fa-42d1-b14c-a43129328c1d-kube-api-access-l9wnn\") pod \"authorino-6d4dfc649f-n4wx2\" (UID: \"141013db-d6fa-42d1-b14c-a43129328c1d\") " pod="kuadrant-system/authorino-6d4dfc649f-n4wx2"
Sep 15 20:16:31.010013 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:31.009841    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48k4p\" (UniqueName: \"kubernetes.io/projected/8781f715-6477-43e5-b369-fa73fe153c96-kube-api-access-48k4p\") pod \"authorino-557f876f69-6h7x7\" (UID: \"8781f715-6477-43e5-b369-fa73fe153c96\") " pod="kuadrant-system/authorino-557f876f69-6h7x7"
Sep 15 20:16:31.010013 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:31.009877    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/141013db-d6fa-42d1-b14c-a43129328c1d-tls-cert\") pod \"authorino-6d4dfc649f-n4wx2\" (UID: \"141013db-d6fa-42d1-b14c-a43129328c1d\") " pod="kuadrant-system/authorino-6d4dfc649f-n4wx2"
Sep 15 20:16:31.018070 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:31.018045    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48k4p\" (UniqueName: \"kubernetes.io/projected/8781f715-6477-43e5-b369-fa73fe153c96-kube-api-access-48k4p\") pod \"authorino-557f876f69-6h7x7\" (UID: \"8781f715-6477-43e5-b369-fa73fe153c96\") " pod="kuadrant-system/authorino-557f876f69-6h7x7"
Sep 15 20:16:31.039119 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:31.039092    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-n4wx2"]
Sep 15 20:16:31.039324 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:16:31.039305    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-l9wnn tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-n4wx2" podUID="141013db-d6fa-42d1-b14c-a43129328c1d"
Sep 15 20:16:31.066857 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:31.066802    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-84574cdcc-bq92v"]
Sep 15 20:16:31.070148 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:31.070131    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84574cdcc-bq92v"
Sep 15 20:16:31.078635 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:31.078615    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-84574cdcc-bq92v"]
Sep 15 20:16:31.111057 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:31.111032    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k6mw\" (UniqueName: \"kubernetes.io/projected/99f49138-c843-40ae-a1cf-631c8d42ef42-kube-api-access-6k6mw\") pod \"authorino-84574cdcc-bq92v\" (UID: \"99f49138-c843-40ae-a1cf-631c8d42ef42\") " pod="kuadrant-system/authorino-84574cdcc-bq92v"
Sep 15 20:16:31.111160 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:31.111079    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9wnn\" (UniqueName: \"kubernetes.io/projected/141013db-d6fa-42d1-b14c-a43129328c1d-kube-api-access-l9wnn\") pod \"authorino-6d4dfc649f-n4wx2\" (UID: \"141013db-d6fa-42d1-b14c-a43129328c1d\") " pod="kuadrant-system/authorino-6d4dfc649f-n4wx2"
Sep 15 20:16:31.111160 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:31.111137    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/99f49138-c843-40ae-a1cf-631c8d42ef42-tls-cert\") pod \"authorino-84574cdcc-bq92v\" (UID: \"99f49138-c843-40ae-a1cf-631c8d42ef42\") " pod="kuadrant-system/authorino-84574cdcc-bq92v"
Sep 15 20:16:31.111277 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:31.111181    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/141013db-d6fa-42d1-b14c-a43129328c1d-tls-cert\") pod \"authorino-6d4dfc649f-n4wx2\" (UID: \"141013db-d6fa-42d1-b14c-a43129328c1d\") " pod="kuadrant-system/authorino-6d4dfc649f-n4wx2"
Sep 15 20:16:31.113544 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:31.113524    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/141013db-d6fa-42d1-b14c-a43129328c1d-tls-cert\") pod \"authorino-6d4dfc649f-n4wx2\" (UID: \"141013db-d6fa-42d1-b14c-a43129328c1d\") " pod="kuadrant-system/authorino-6d4dfc649f-n4wx2"
Sep 15 20:16:31.119574 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:31.119550    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9wnn\" (UniqueName: \"kubernetes.io/projected/141013db-d6fa-42d1-b14c-a43129328c1d-kube-api-access-l9wnn\") pod \"authorino-6d4dfc649f-n4wx2\" (UID: \"141013db-d6fa-42d1-b14c-a43129328c1d\") " pod="kuadrant-system/authorino-6d4dfc649f-n4wx2"
Sep 15 20:16:31.211956 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:31.211908    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k6mw\" (UniqueName: \"kubernetes.io/projected/99f49138-c843-40ae-a1cf-631c8d42ef42-kube-api-access-6k6mw\") pod \"authorino-84574cdcc-bq92v\" (UID: \"99f49138-c843-40ae-a1cf-631c8d42ef42\") " pod="kuadrant-system/authorino-84574cdcc-bq92v"
Sep 15 20:16:31.212056 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:31.211996    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/99f49138-c843-40ae-a1cf-631c8d42ef42-tls-cert\") pod \"authorino-84574cdcc-bq92v\" (UID: \"99f49138-c843-40ae-a1cf-631c8d42ef42\") " pod="kuadrant-system/authorino-84574cdcc-bq92v"
Sep 15 20:16:31.214173 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:31.214153    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/99f49138-c843-40ae-a1cf-631c8d42ef42-tls-cert\") pod \"authorino-84574cdcc-bq92v\" (UID: \"99f49138-c843-40ae-a1cf-631c8d42ef42\") " pod="kuadrant-system/authorino-84574cdcc-bq92v"
Sep 15 20:16:31.220147 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:31.220127    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k6mw\" (UniqueName: \"kubernetes.io/projected/99f49138-c843-40ae-a1cf-631c8d42ef42-kube-api-access-6k6mw\") pod \"authorino-84574cdcc-bq92v\" (UID: \"99f49138-c843-40ae-a1cf-631c8d42ef42\") " pod="kuadrant-system/authorino-84574cdcc-bq92v"
Sep 15 20:16:31.379090 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:31.379005    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84574cdcc-bq92v"
Sep 15 20:16:31.644798 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:31.644750    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-84574cdcc-bq92v"]
Sep 15 20:16:31.647068 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:16:31.647042    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod99f49138_c843_40ae_a1cf_631c8d42ef42.slice/crio-ac46caff0e08e4094a72bf4563810592cca7df4834a449f77e02ce10c7f007e9 WatchSource:0}: Error finding container ac46caff0e08e4094a72bf4563810592cca7df4834a449f77e02ce10c7f007e9: Status 404 returned error can't find the container with id ac46caff0e08e4094a72bf4563810592cca7df4834a449f77e02ce10c7f007e9
Sep 15 20:16:31.810783 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:31.810758    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-6h7x7"
Sep 15 20:16:31.810783 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:31.810778    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84574cdcc-bq92v" event={"ID":"99f49138-c843-40ae-a1cf-631c8d42ef42","Type":"ContainerStarted","Data":"ac46caff0e08e4094a72bf4563810592cca7df4834a449f77e02ce10c7f007e9"}
Sep 15 20:16:31.810987 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:31.810911    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-n4wx2"
Sep 15 20:16:31.818752 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:31.818731    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-6h7x7"
Sep 15 20:16:31.820683 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:31.820666    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-n4wx2"
Sep 15 20:16:31.916517 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:31.916439    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l9wnn\" (UniqueName: \"kubernetes.io/projected/141013db-d6fa-42d1-b14c-a43129328c1d-kube-api-access-l9wnn\") pod \"141013db-d6fa-42d1-b14c-a43129328c1d\" (UID: \"141013db-d6fa-42d1-b14c-a43129328c1d\") "
Sep 15 20:16:31.916517 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:31.916503    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48k4p\" (UniqueName: \"kubernetes.io/projected/8781f715-6477-43e5-b369-fa73fe153c96-kube-api-access-48k4p\") pod \"8781f715-6477-43e5-b369-fa73fe153c96\" (UID: \"8781f715-6477-43e5-b369-fa73fe153c96\") "
Sep 15 20:16:31.916721 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:31.916618    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/141013db-d6fa-42d1-b14c-a43129328c1d-tls-cert\") pod \"141013db-d6fa-42d1-b14c-a43129328c1d\" (UID: \"141013db-d6fa-42d1-b14c-a43129328c1d\") "
Sep 15 20:16:31.918731 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:31.918703    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/141013db-d6fa-42d1-b14c-a43129328c1d-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "141013db-d6fa-42d1-b14c-a43129328c1d" (UID: "141013db-d6fa-42d1-b14c-a43129328c1d"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:16:31.918731 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:31.918714    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/141013db-d6fa-42d1-b14c-a43129328c1d-kube-api-access-l9wnn" (OuterVolumeSpecName: "kube-api-access-l9wnn") pod "141013db-d6fa-42d1-b14c-a43129328c1d" (UID: "141013db-d6fa-42d1-b14c-a43129328c1d"). InnerVolumeSpecName "kube-api-access-l9wnn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:16:31.918947 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:31.918758    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8781f715-6477-43e5-b369-fa73fe153c96-kube-api-access-48k4p" (OuterVolumeSpecName: "kube-api-access-48k4p") pod "8781f715-6477-43e5-b369-fa73fe153c96" (UID: "8781f715-6477-43e5-b369-fa73fe153c96"). InnerVolumeSpecName "kube-api-access-48k4p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:16:32.018258 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:32.018152    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l9wnn\" (UniqueName: \"kubernetes.io/projected/141013db-d6fa-42d1-b14c-a43129328c1d-kube-api-access-l9wnn\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:16:32.018258 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:32.018183    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48k4p\" (UniqueName: \"kubernetes.io/projected/8781f715-6477-43e5-b369-fa73fe153c96-kube-api-access-48k4p\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:16:32.018258 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:32.018209    2576 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/141013db-d6fa-42d1-b14c-a43129328c1d-tls-cert\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:16:32.817343 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:32.817315    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-6h7x7"
Sep 15 20:16:32.817343 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:32.817329    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84574cdcc-bq92v" event={"ID":"99f49138-c843-40ae-a1cf-631c8d42ef42","Type":"ContainerStarted","Data":"15b1ac9ed7134f61b2e414da905ab3519dce96fd734f8b6a3ea129c4020f3343"}
Sep 15 20:16:32.817863 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:32.817480    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-n4wx2"
Sep 15 20:16:32.854933 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:32.854863    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-84574cdcc-bq92v" podStartSLOduration=1.572498037 podStartE2EDuration="1.854843266s" totalImagesPullingTime="282.345229ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:16:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:16:32.833327943 +0000 UTC m=+707.321792424" watchObservedRunningTime="2026-09-15 20:16:32.854843266 +0000 UTC m=+707.343307742"
Sep 15 20:16:32.858471 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:32.858447    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-cx2j5"]
Sep 15 20:16:32.858662 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:32.858641    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-cx2j5" podUID="3c314d5f-1209-4674-ada6-dd67e4152142" containerName="authorino" containerID="cri-o://b4d52d6874a9e5c3f24baa38037d4c02f73ae9f61c142716ac59e09a5e10e6fd" gracePeriod=30
Sep 15 20:16:32.864548 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:32.864523    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-n4wx2"]
Sep 15 20:16:32.872232 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:32.872206    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-n4wx2"]
Sep 15 20:16:32.891262 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:32.891238    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-6h7x7"]
Sep 15 20:16:32.894699 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:32.894676    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-6h7x7"]
Sep 15 20:16:33.096262 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:33.096236    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-cx2j5"
Sep 15 20:16:33.128597 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:33.128572    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8zmpl\" (UniqueName: \"kubernetes.io/projected/3c314d5f-1209-4674-ada6-dd67e4152142-kube-api-access-8zmpl\") pod \"3c314d5f-1209-4674-ada6-dd67e4152142\" (UID: \"3c314d5f-1209-4674-ada6-dd67e4152142\") "
Sep 15 20:16:33.130582 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:33.130559    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c314d5f-1209-4674-ada6-dd67e4152142-kube-api-access-8zmpl" (OuterVolumeSpecName: "kube-api-access-8zmpl") pod "3c314d5f-1209-4674-ada6-dd67e4152142" (UID: "3c314d5f-1209-4674-ada6-dd67e4152142"). InnerVolumeSpecName "kube-api-access-8zmpl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:16:33.229867 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:33.229828    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8zmpl\" (UniqueName: \"kubernetes.io/projected/3c314d5f-1209-4674-ada6-dd67e4152142-kube-api-access-8zmpl\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:16:33.823604 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:33.823575    2576 generic.go:356] "Generic (PLEG): container finished" podID="3c314d5f-1209-4674-ada6-dd67e4152142" containerID="b4d52d6874a9e5c3f24baa38037d4c02f73ae9f61c142716ac59e09a5e10e6fd" exitCode=0
Sep 15 20:16:33.824037 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:33.823614    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-cx2j5"
Sep 15 20:16:33.824037 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:33.823663    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-cx2j5" event={"ID":"3c314d5f-1209-4674-ada6-dd67e4152142","Type":"ContainerDied","Data":"b4d52d6874a9e5c3f24baa38037d4c02f73ae9f61c142716ac59e09a5e10e6fd"}
Sep 15 20:16:33.824037 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:33.823707    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-cx2j5" event={"ID":"3c314d5f-1209-4674-ada6-dd67e4152142","Type":"ContainerDied","Data":"4f0d27038d22ba944cc32d934408fba62f25a59f6d8e06d6bfe8ff87991c07d9"}
Sep 15 20:16:33.824037 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:33.823725    2576 scope.go:117] "RemoveContainer" containerID="b4d52d6874a9e5c3f24baa38037d4c02f73ae9f61c142716ac59e09a5e10e6fd"
Sep 15 20:16:33.832016 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:33.832000    2576 scope.go:117] "RemoveContainer" containerID="b4d52d6874a9e5c3f24baa38037d4c02f73ae9f61c142716ac59e09a5e10e6fd"
Sep 15 20:16:33.832267 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:16:33.832249    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4d52d6874a9e5c3f24baa38037d4c02f73ae9f61c142716ac59e09a5e10e6fd\": container with ID starting with b4d52d6874a9e5c3f24baa38037d4c02f73ae9f61c142716ac59e09a5e10e6fd not found: ID does not exist" containerID="b4d52d6874a9e5c3f24baa38037d4c02f73ae9f61c142716ac59e09a5e10e6fd"
Sep 15 20:16:33.832322 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:33.832273    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4d52d6874a9e5c3f24baa38037d4c02f73ae9f61c142716ac59e09a5e10e6fd"} err="failed to get container status \"b4d52d6874a9e5c3f24baa38037d4c02f73ae9f61c142716ac59e09a5e10e6fd\": rpc error: code = NotFound desc = could not find container \"b4d52d6874a9e5c3f24baa38037d4c02f73ae9f61c142716ac59e09a5e10e6fd\": container with ID starting with b4d52d6874a9e5c3f24baa38037d4c02f73ae9f61c142716ac59e09a5e10e6fd not found: ID does not exist"
Sep 15 20:16:33.844804 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:33.844781    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-cx2j5"]
Sep 15 20:16:33.859345 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:33.859322    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-cx2j5"]
Sep 15 20:16:34.143317 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:34.143227    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="141013db-d6fa-42d1-b14c-a43129328c1d" path="/var/lib/kubelet/pods/141013db-d6fa-42d1-b14c-a43129328c1d/volumes"
Sep 15 20:16:34.143544 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:34.143526    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c314d5f-1209-4674-ada6-dd67e4152142" path="/var/lib/kubelet/pods/3c314d5f-1209-4674-ada6-dd67e4152142/volumes"
Sep 15 20:16:34.143872 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:34.143855    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8781f715-6477-43e5-b369-fa73fe153c96" path="/var/lib/kubelet/pods/8781f715-6477-43e5-b369-fa73fe153c96/volumes"
Sep 15 20:16:37.778341 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:37.778314    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:16:37.876354 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:37.876323    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:16:37.939167 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:37.939142    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:16:38.029071 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:38.028993    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:16:41.217398 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:41.217366    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:16:41.307204 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:16:41.307176    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:17:11.205249 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:11.205216    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:17:11.302169 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:11.302141    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:17:12.768544 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:12.768508    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:17:12.879463 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:12.879421    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:17:12.986692 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:12.986659    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:17:13.095439 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:13.095354    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:17:13.379167 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:13.379094    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:17:13.467427 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:13.467395    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:17:13.569747 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:13.569716    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-869x2"]
Sep 15 20:17:13.570057 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:13.570045    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c314d5f-1209-4674-ada6-dd67e4152142" containerName="authorino"
Sep 15 20:17:13.570118 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:13.570058    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c314d5f-1209-4674-ada6-dd67e4152142" containerName="authorino"
Sep 15 20:17:13.570161 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:13.570125    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c314d5f-1209-4674-ada6-dd67e4152142" containerName="authorino"
Sep 15 20:17:13.572881 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:13.572865    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-869x2"
Sep 15 20:17:13.575214 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:13.575193    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-hjmnc\""
Sep 15 20:17:13.575214 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:13.575212    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 15 20:17:13.581691 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:13.581671    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-869x2"]
Sep 15 20:17:13.652912 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:13.652825    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0599d238-3847-4c55-89c3-c3074e1d9ad0-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-869x2\" (UID: \"0599d238-3847-4c55-89c3-c3074e1d9ad0\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-869x2"
Sep 15 20:17:13.652912 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:13.652884    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bd2bl\" (UniqueName: \"kubernetes.io/projected/0599d238-3847-4c55-89c3-c3074e1d9ad0-kube-api-access-bd2bl\") pod \"payload-pre-processing-59fbc47bf8-869x2\" (UID: \"0599d238-3847-4c55-89c3-c3074e1d9ad0\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-869x2"
Sep 15 20:17:13.753827 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:13.753787    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0599d238-3847-4c55-89c3-c3074e1d9ad0-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-869x2\" (UID: \"0599d238-3847-4c55-89c3-c3074e1d9ad0\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-869x2"
Sep 15 20:17:13.753827 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:13.753832    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bd2bl\" (UniqueName: \"kubernetes.io/projected/0599d238-3847-4c55-89c3-c3074e1d9ad0-kube-api-access-bd2bl\") pod \"payload-pre-processing-59fbc47bf8-869x2\" (UID: \"0599d238-3847-4c55-89c3-c3074e1d9ad0\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-869x2"
Sep 15 20:17:13.755100 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:13.755049    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0599d238-3847-4c55-89c3-c3074e1d9ad0-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-869x2\" (UID: \"0599d238-3847-4c55-89c3-c3074e1d9ad0\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-869x2"
Sep 15 20:17:13.771358 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:13.771332    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bd2bl\" (UniqueName: \"kubernetes.io/projected/0599d238-3847-4c55-89c3-c3074e1d9ad0-kube-api-access-bd2bl\") pod \"payload-pre-processing-59fbc47bf8-869x2\" (UID: \"0599d238-3847-4c55-89c3-c3074e1d9ad0\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-869x2"
Sep 15 20:17:13.884296 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:13.884247    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-869x2"
Sep 15 20:17:14.152067 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:14.151996    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-869x2"]
Sep 15 20:17:14.154192 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:17:14.154166    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0599d238_3847_4c55_89c3_c3074e1d9ad0.slice/crio-15ebc4a54a5469fa8d2f53e87c7cb7c46aef56fa2c9532ed84d3671b8b3546b1 WatchSource:0}: Error finding container 15ebc4a54a5469fa8d2f53e87c7cb7c46aef56fa2c9532ed84d3671b8b3546b1: Status 404 returned error can't find the container with id 15ebc4a54a5469fa8d2f53e87c7cb7c46aef56fa2c9532ed84d3671b8b3546b1
Sep 15 20:17:15.012241 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:15.012195    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-869x2" event={"ID":"0599d238-3847-4c55-89c3-c3074e1d9ad0","Type":"ContainerStarted","Data":"15ebc4a54a5469fa8d2f53e87c7cb7c46aef56fa2c9532ed84d3671b8b3546b1"}
Sep 15 20:17:17.025111 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:17.025074    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-869x2" event={"ID":"0599d238-3847-4c55-89c3-c3074e1d9ad0","Type":"ContainerStarted","Data":"ced35a1efe74f6f1c8d1e399be14400e4033f0c2b55c02f71809d4033a9da295"}
Sep 15 20:17:17.025473 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:17.025167    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-869x2"
Sep 15 20:17:23.580263 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:23.580235    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:17:23.685739 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:23.685703    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:17:28.032606 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:28.032569    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-869x2"
Sep 15 20:17:28.051822 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:28.051775    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-869x2" podStartSLOduration=12.758004912 podStartE2EDuration="15.051759959s" totalImagesPullingTime="2.293755047s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:17:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:17:17.04812082 +0000 UTC m=+751.536585298" watchObservedRunningTime="2026-09-15 20:17:28.051759959 +0000 UTC m=+762.540224435"
Sep 15 20:17:46.542974 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:46.542936    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s"]
Sep 15 20:17:46.546548 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:46.546524    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s"
Sep 15 20:17:46.550500 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:46.550473    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mtfkw\""
Sep 15 20:17:46.550500 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:46.550497    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 15 20:17:46.550693 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:46.550530    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 15 20:17:46.550693 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:46.550480    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 15 20:17:46.556902 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:46.556883    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s"]
Sep 15 20:17:46.610337 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:46.610314    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp76r\" (UniqueName: \"kubernetes.io/projected/52855211-f580-4ab1-87b7-edbf73326b5c-kube-api-access-kp76r\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s\" (UID: \"52855211-f580-4ab1-87b7-edbf73326b5c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s"
Sep 15 20:17:46.610466 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:46.610346    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52855211-f580-4ab1-87b7-edbf73326b5c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s\" (UID: \"52855211-f580-4ab1-87b7-edbf73326b5c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s"
Sep 15 20:17:46.610466 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:46.610368    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52855211-f580-4ab1-87b7-edbf73326b5c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s\" (UID: \"52855211-f580-4ab1-87b7-edbf73326b5c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s"
Sep 15 20:17:46.610466 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:46.610395    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52855211-f580-4ab1-87b7-edbf73326b5c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s\" (UID: \"52855211-f580-4ab1-87b7-edbf73326b5c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s"
Sep 15 20:17:46.610466 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:46.610444    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52855211-f580-4ab1-87b7-edbf73326b5c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s\" (UID: \"52855211-f580-4ab1-87b7-edbf73326b5c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s"
Sep 15 20:17:46.610603 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:46.610481    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52855211-f580-4ab1-87b7-edbf73326b5c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s\" (UID: \"52855211-f580-4ab1-87b7-edbf73326b5c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s"
Sep 15 20:17:46.711160 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:46.711127    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52855211-f580-4ab1-87b7-edbf73326b5c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s\" (UID: \"52855211-f580-4ab1-87b7-edbf73326b5c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s"
Sep 15 20:17:46.711160 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:46.711165    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52855211-f580-4ab1-87b7-edbf73326b5c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s\" (UID: \"52855211-f580-4ab1-87b7-edbf73326b5c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s"
Sep 15 20:17:46.711378 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:46.711186    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52855211-f580-4ab1-87b7-edbf73326b5c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s\" (UID: \"52855211-f580-4ab1-87b7-edbf73326b5c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s"
Sep 15 20:17:46.711378 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:46.711211    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52855211-f580-4ab1-87b7-edbf73326b5c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s\" (UID: \"52855211-f580-4ab1-87b7-edbf73326b5c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s"
Sep 15 20:17:46.711378 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:46.711244    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52855211-f580-4ab1-87b7-edbf73326b5c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s\" (UID: \"52855211-f580-4ab1-87b7-edbf73326b5c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s"
Sep 15 20:17:46.711378 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:46.711281    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp76r\" (UniqueName: \"kubernetes.io/projected/52855211-f580-4ab1-87b7-edbf73326b5c-kube-api-access-kp76r\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s\" (UID: \"52855211-f580-4ab1-87b7-edbf73326b5c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s"
Sep 15 20:17:46.711622 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:46.711598    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52855211-f580-4ab1-87b7-edbf73326b5c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s\" (UID: \"52855211-f580-4ab1-87b7-edbf73326b5c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s"
Sep 15 20:17:46.711674 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:46.711622    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52855211-f580-4ab1-87b7-edbf73326b5c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s\" (UID: \"52855211-f580-4ab1-87b7-edbf73326b5c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s"
Sep 15 20:17:46.711710 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:46.711671    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52855211-f580-4ab1-87b7-edbf73326b5c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s\" (UID: \"52855211-f580-4ab1-87b7-edbf73326b5c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s"
Sep 15 20:17:46.713474 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:46.713454    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52855211-f580-4ab1-87b7-edbf73326b5c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s\" (UID: \"52855211-f580-4ab1-87b7-edbf73326b5c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s"
Sep 15 20:17:46.713657 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:46.713639    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52855211-f580-4ab1-87b7-edbf73326b5c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s\" (UID: \"52855211-f580-4ab1-87b7-edbf73326b5c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s"
Sep 15 20:17:46.716236 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:46.716214    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:17:46.723447 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:46.723427    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp76r\" (UniqueName: \"kubernetes.io/projected/52855211-f580-4ab1-87b7-edbf73326b5c-kube-api-access-kp76r\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s\" (UID: \"52855211-f580-4ab1-87b7-edbf73326b5c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s"
Sep 15 20:17:46.857045 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:46.856974    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s"
Sep 15 20:17:47.142740 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:47.142661    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s"]
Sep 15 20:17:47.145543 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:17:47.145514    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52855211_f580_4ab1_87b7_edbf73326b5c.slice/crio-f3dddd7f94db68f8f2ac2568c3fe8b93973ad0ebbed4d77cecfa5e15b6611d0b WatchSource:0}: Error finding container f3dddd7f94db68f8f2ac2568c3fe8b93973ad0ebbed4d77cecfa5e15b6611d0b: Status 404 returned error can't find the container with id f3dddd7f94db68f8f2ac2568c3fe8b93973ad0ebbed4d77cecfa5e15b6611d0b
Sep 15 20:17:47.161940 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:47.161889    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s" event={"ID":"52855211-f580-4ab1-87b7-edbf73326b5c","Type":"ContainerStarted","Data":"f3dddd7f94db68f8f2ac2568c3fe8b93973ad0ebbed4d77cecfa5e15b6611d0b"}
Sep 15 20:17:49.172009 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:49.171978    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s" event={"ID":"52855211-f580-4ab1-87b7-edbf73326b5c","Type":"ContainerStarted","Data":"e5ff9e34e4b5544a1cc43cde164e6a95c45ecac90fb66370cf462dbf2aabddc2"}
Sep 15 20:17:49.188725 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:49.188675    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s" podStartSLOduration=1.939588222 podStartE2EDuration="3.188660157s" totalImagesPullingTime="1.249071935s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:17:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:17:49.187218045 +0000 UTC m=+783.675682521" watchObservedRunningTime="2026-09-15 20:17:49.188660157 +0000 UTC m=+783.677124633"
Sep 15 20:17:56.857806 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:56.857763    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s"
Sep 15 20:17:56.858213 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:56.857827    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s"
Sep 15 20:17:56.870356 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:56.870332    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s"
Sep 15 20:17:57.215551 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:57.215525    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s"
Sep 15 20:17:57.958648 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:17:57.958605    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:18:04.779383 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:04.779348    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:18:40.261065 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:40.261034    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8vtfv"]
Sep 15 20:18:40.264320 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:40.264300    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8vtfv"
Sep 15 20:18:40.267104 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:40.267083    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 15 20:18:40.277735 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:40.277713    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8vtfv"]
Sep 15 20:18:40.290759 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:40.290733    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:18:40.341664 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:40.341632    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0bfa23e-9b3f-42cf-b040-1c97b27fbe69-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8vtfv\" (UID: \"e0bfa23e-9b3f-42cf-b040-1c97b27fbe69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8vtfv"
Sep 15 20:18:40.341808 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:40.341700    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tc7px\" (UniqueName: \"kubernetes.io/projected/e0bfa23e-9b3f-42cf-b040-1c97b27fbe69-kube-api-access-tc7px\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8vtfv\" (UID: \"e0bfa23e-9b3f-42cf-b040-1c97b27fbe69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8vtfv"
Sep 15 20:18:40.341808 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:40.341733    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0bfa23e-9b3f-42cf-b040-1c97b27fbe69-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8vtfv\" (UID: \"e0bfa23e-9b3f-42cf-b040-1c97b27fbe69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8vtfv"
Sep 15 20:18:40.341808 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:40.341777    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0bfa23e-9b3f-42cf-b040-1c97b27fbe69-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8vtfv\" (UID: \"e0bfa23e-9b3f-42cf-b040-1c97b27fbe69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8vtfv"
Sep 15 20:18:40.342013 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:40.341849    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0bfa23e-9b3f-42cf-b040-1c97b27fbe69-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8vtfv\" (UID: \"e0bfa23e-9b3f-42cf-b040-1c97b27fbe69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8vtfv"
Sep 15 20:18:40.342013 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:40.341891    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0bfa23e-9b3f-42cf-b040-1c97b27fbe69-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8vtfv\" (UID: \"e0bfa23e-9b3f-42cf-b040-1c97b27fbe69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8vtfv"
Sep 15 20:18:40.442751 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:40.442721    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0bfa23e-9b3f-42cf-b040-1c97b27fbe69-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8vtfv\" (UID: \"e0bfa23e-9b3f-42cf-b040-1c97b27fbe69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8vtfv"
Sep 15 20:18:40.442911 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:40.442758    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0bfa23e-9b3f-42cf-b040-1c97b27fbe69-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8vtfv\" (UID: \"e0bfa23e-9b3f-42cf-b040-1c97b27fbe69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8vtfv"
Sep 15 20:18:40.442911 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:40.442776    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0bfa23e-9b3f-42cf-b040-1c97b27fbe69-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8vtfv\" (UID: \"e0bfa23e-9b3f-42cf-b040-1c97b27fbe69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8vtfv"
Sep 15 20:18:40.442911 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:40.442812    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tc7px\" (UniqueName: \"kubernetes.io/projected/e0bfa23e-9b3f-42cf-b040-1c97b27fbe69-kube-api-access-tc7px\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8vtfv\" (UID: \"e0bfa23e-9b3f-42cf-b040-1c97b27fbe69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8vtfv"
Sep 15 20:18:40.442911 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:40.442833    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0bfa23e-9b3f-42cf-b040-1c97b27fbe69-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8vtfv\" (UID: \"e0bfa23e-9b3f-42cf-b040-1c97b27fbe69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8vtfv"
Sep 15 20:18:40.442911 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:40.442858    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0bfa23e-9b3f-42cf-b040-1c97b27fbe69-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8vtfv\" (UID: \"e0bfa23e-9b3f-42cf-b040-1c97b27fbe69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8vtfv"
Sep 15 20:18:40.443214 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:40.443104    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0bfa23e-9b3f-42cf-b040-1c97b27fbe69-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8vtfv\" (UID: \"e0bfa23e-9b3f-42cf-b040-1c97b27fbe69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8vtfv"
Sep 15 20:18:40.443214 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:40.443174    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0bfa23e-9b3f-42cf-b040-1c97b27fbe69-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8vtfv\" (UID: \"e0bfa23e-9b3f-42cf-b040-1c97b27fbe69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8vtfv"
Sep 15 20:18:40.443298 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:40.443247    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0bfa23e-9b3f-42cf-b040-1c97b27fbe69-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8vtfv\" (UID: \"e0bfa23e-9b3f-42cf-b040-1c97b27fbe69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8vtfv"
Sep 15 20:18:40.445078 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:40.445052    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0bfa23e-9b3f-42cf-b040-1c97b27fbe69-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8vtfv\" (UID: \"e0bfa23e-9b3f-42cf-b040-1c97b27fbe69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8vtfv"
Sep 15 20:18:40.445204 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:40.445187    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0bfa23e-9b3f-42cf-b040-1c97b27fbe69-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8vtfv\" (UID: \"e0bfa23e-9b3f-42cf-b040-1c97b27fbe69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8vtfv"
Sep 15 20:18:40.450580 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:40.450557    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tc7px\" (UniqueName: \"kubernetes.io/projected/e0bfa23e-9b3f-42cf-b040-1c97b27fbe69-kube-api-access-tc7px\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8vtfv\" (UID: \"e0bfa23e-9b3f-42cf-b040-1c97b27fbe69\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8vtfv"
Sep 15 20:18:40.574946 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:40.574873    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8vtfv"
Sep 15 20:18:40.842564 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:40.842535    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8vtfv"]
Sep 15 20:18:40.844740 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:18:40.844715    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0bfa23e_9b3f_42cf_b040_1c97b27fbe69.slice/crio-bf814fc1c58b267ceb86b97f1f0625dae1667d1d42232689d5006a8823d4f666 WatchSource:0}: Error finding container bf814fc1c58b267ceb86b97f1f0625dae1667d1d42232689d5006a8823d4f666: Status 404 returned error can't find the container with id bf814fc1c58b267ceb86b97f1f0625dae1667d1d42232689d5006a8823d4f666
Sep 15 20:18:41.411972 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:41.411935    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8vtfv" event={"ID":"e0bfa23e-9b3f-42cf-b040-1c97b27fbe69","Type":"ContainerStarted","Data":"ab1f0b9e3e48334adcb5b49781981d059c59a0f1a743c6373b0acf98c1dce882"}
Sep 15 20:18:41.412342 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:41.411996    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8vtfv" event={"ID":"e0bfa23e-9b3f-42cf-b040-1c97b27fbe69","Type":"ContainerStarted","Data":"bf814fc1c58b267ceb86b97f1f0625dae1667d1d42232689d5006a8823d4f666"}
Sep 15 20:18:41.430410 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:41.430360    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8vtfv" podStartSLOduration=1.244996485 podStartE2EDuration="1.43034675s" totalImagesPullingTime="185.350265ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:18:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:18:41.429271472 +0000 UTC m=+835.917735948" watchObservedRunningTime="2026-09-15 20:18:41.43034675 +0000 UTC m=+835.918811226"
Sep 15 20:18:50.575638 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:50.575592    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8vtfv"
Sep 15 20:18:50.576093 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:50.575659    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8vtfv"
Sep 15 20:18:50.590648 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:50.590620    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8vtfv"
Sep 15 20:18:51.466144 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:51.466117    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8vtfv"
Sep 15 20:18:55.970121 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:55.970085    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:18:56.026842 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:18:56.026815    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:19:00.725603 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:00.725566    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:19:03.499174 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:03.499141    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:19:03.535101 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:03.535071    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-2bd8x"]
Sep 15 20:19:03.538618 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:03.538603    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2bd8x"
Sep 15 20:19:03.540950 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:03.540900    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 15 20:19:03.548061 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:03.548037    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-2bd8x"]
Sep 15 20:19:03.566560 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:03.566540    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:19:03.619199 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:03.619171    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:19:03.642371 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:03.642349    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0629cd62-1aae-406d-9ce1-d8e6e964a624-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2bd8x\" (UID: \"0629cd62-1aae-406d-9ce1-d8e6e964a624\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2bd8x"
Sep 15 20:19:03.642443 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:03.642407    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0629cd62-1aae-406d-9ce1-d8e6e964a624-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2bd8x\" (UID: \"0629cd62-1aae-406d-9ce1-d8e6e964a624\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2bd8x"
Sep 15 20:19:03.642487 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:03.642465    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4tff\" (UniqueName: \"kubernetes.io/projected/0629cd62-1aae-406d-9ce1-d8e6e964a624-kube-api-access-p4tff\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2bd8x\" (UID: \"0629cd62-1aae-406d-9ce1-d8e6e964a624\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2bd8x"
Sep 15 20:19:03.642524 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:03.642495    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0629cd62-1aae-406d-9ce1-d8e6e964a624-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2bd8x\" (UID: \"0629cd62-1aae-406d-9ce1-d8e6e964a624\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2bd8x"
Sep 15 20:19:03.642562 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:03.642549    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0629cd62-1aae-406d-9ce1-d8e6e964a624-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2bd8x\" (UID: \"0629cd62-1aae-406d-9ce1-d8e6e964a624\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2bd8x"
Sep 15 20:19:03.642595 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:03.642572    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0629cd62-1aae-406d-9ce1-d8e6e964a624-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2bd8x\" (UID: \"0629cd62-1aae-406d-9ce1-d8e6e964a624\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2bd8x"
Sep 15 20:19:03.743356 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:03.743325    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0629cd62-1aae-406d-9ce1-d8e6e964a624-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2bd8x\" (UID: \"0629cd62-1aae-406d-9ce1-d8e6e964a624\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2bd8x"
Sep 15 20:19:03.743472 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:03.743358    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4tff\" (UniqueName: \"kubernetes.io/projected/0629cd62-1aae-406d-9ce1-d8e6e964a624-kube-api-access-p4tff\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2bd8x\" (UID: \"0629cd62-1aae-406d-9ce1-d8e6e964a624\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2bd8x"
Sep 15 20:19:03.743472 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:03.743376    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0629cd62-1aae-406d-9ce1-d8e6e964a624-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2bd8x\" (UID: \"0629cd62-1aae-406d-9ce1-d8e6e964a624\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2bd8x"
Sep 15 20:19:03.743472 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:03.743400    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0629cd62-1aae-406d-9ce1-d8e6e964a624-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2bd8x\" (UID: \"0629cd62-1aae-406d-9ce1-d8e6e964a624\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2bd8x"
Sep 15 20:19:03.743627 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:03.743600    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0629cd62-1aae-406d-9ce1-d8e6e964a624-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2bd8x\" (UID: \"0629cd62-1aae-406d-9ce1-d8e6e964a624\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2bd8x"
Sep 15 20:19:03.743709 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:03.743695    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0629cd62-1aae-406d-9ce1-d8e6e964a624-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2bd8x\" (UID: \"0629cd62-1aae-406d-9ce1-d8e6e964a624\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2bd8x"
Sep 15 20:19:03.743784 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:03.743766    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0629cd62-1aae-406d-9ce1-d8e6e964a624-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2bd8x\" (UID: \"0629cd62-1aae-406d-9ce1-d8e6e964a624\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2bd8x"
Sep 15 20:19:03.744003 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:03.743984    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0629cd62-1aae-406d-9ce1-d8e6e964a624-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2bd8x\" (UID: \"0629cd62-1aae-406d-9ce1-d8e6e964a624\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2bd8x"
Sep 15 20:19:03.744087 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:03.744042    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0629cd62-1aae-406d-9ce1-d8e6e964a624-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2bd8x\" (UID: \"0629cd62-1aae-406d-9ce1-d8e6e964a624\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2bd8x"
Sep 15 20:19:03.745613 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:03.745595    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0629cd62-1aae-406d-9ce1-d8e6e964a624-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2bd8x\" (UID: \"0629cd62-1aae-406d-9ce1-d8e6e964a624\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2bd8x"
Sep 15 20:19:03.745931 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:03.745899    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0629cd62-1aae-406d-9ce1-d8e6e964a624-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2bd8x\" (UID: \"0629cd62-1aae-406d-9ce1-d8e6e964a624\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2bd8x"
Sep 15 20:19:03.754251 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:03.754207    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4tff\" (UniqueName: \"kubernetes.io/projected/0629cd62-1aae-406d-9ce1-d8e6e964a624-kube-api-access-p4tff\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2bd8x\" (UID: \"0629cd62-1aae-406d-9ce1-d8e6e964a624\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2bd8x"
Sep 15 20:19:03.848851 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:03.848820    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2bd8x"
Sep 15 20:19:04.118784 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:04.118758    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-2bd8x"]
Sep 15 20:19:04.121018 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:19:04.120989    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0629cd62_1aae_406d_9ce1_d8e6e964a624.slice/crio-96247ffb5266d96b999f108fb91c94ed528184a68aed7dbb67e9aab1c1db9b06 WatchSource:0}: Error finding container 96247ffb5266d96b999f108fb91c94ed528184a68aed7dbb67e9aab1c1db9b06: Status 404 returned error can't find the container with id 96247ffb5266d96b999f108fb91c94ed528184a68aed7dbb67e9aab1c1db9b06
Sep 15 20:19:04.515189 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:04.515156    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2bd8x" event={"ID":"0629cd62-1aae-406d-9ce1-d8e6e964a624","Type":"ContainerStarted","Data":"7d21cc4e1aa17c66c5490e450a5b32e66ea2766dc6ef3db8aff9a1d112d3e6c7"}
Sep 15 20:19:04.515580 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:04.515208    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2bd8x" event={"ID":"0629cd62-1aae-406d-9ce1-d8e6e964a624","Type":"ContainerStarted","Data":"96247ffb5266d96b999f108fb91c94ed528184a68aed7dbb67e9aab1c1db9b06"}
Sep 15 20:19:04.534516 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:04.534465    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2bd8x" podStartSLOduration=1.338837311 podStartE2EDuration="1.534445584s" totalImagesPullingTime="195.608273ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:19:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:19:04.532476777 +0000 UTC m=+859.020941255" watchObservedRunningTime="2026-09-15 20:19:04.534445584 +0000 UTC m=+859.022910061"
Sep 15 20:19:13.849348 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:13.849313    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2bd8x"
Sep 15 20:19:13.849972 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:13.849379    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2bd8x"
Sep 15 20:19:13.861574 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:13.861551    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2bd8x"
Sep 15 20:19:14.571461 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:14.571430    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2bd8x"
Sep 15 20:19:18.029674 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:18.029639    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m"]
Sep 15 20:19:18.032912 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:18.032891    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m"
Sep 15 20:19:18.034968 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:18.034950    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 15 20:19:18.041636 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:18.041616    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m"]
Sep 15 20:19:18.063984 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:18.063959    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81d98002-64a0-4881-b58a-5eba5848ff97-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m\" (UID: \"81d98002-64a0-4881-b58a-5eba5848ff97\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m"
Sep 15 20:19:18.064092 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:18.064041    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81d98002-64a0-4881-b58a-5eba5848ff97-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m\" (UID: \"81d98002-64a0-4881-b58a-5eba5848ff97\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m"
Sep 15 20:19:18.064092 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:18.064082    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81d98002-64a0-4881-b58a-5eba5848ff97-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m\" (UID: \"81d98002-64a0-4881-b58a-5eba5848ff97\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m"
Sep 15 20:19:18.064201 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:18.064138    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxgf9\" (UniqueName: \"kubernetes.io/projected/81d98002-64a0-4881-b58a-5eba5848ff97-kube-api-access-bxgf9\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m\" (UID: \"81d98002-64a0-4881-b58a-5eba5848ff97\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m"
Sep 15 20:19:18.064245 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:18.064216    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81d98002-64a0-4881-b58a-5eba5848ff97-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m\" (UID: \"81d98002-64a0-4881-b58a-5eba5848ff97\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m"
Sep 15 20:19:18.064283 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:18.064258    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81d98002-64a0-4881-b58a-5eba5848ff97-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m\" (UID: \"81d98002-64a0-4881-b58a-5eba5848ff97\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m"
Sep 15 20:19:18.129282 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:18.129250    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:19:18.165166 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:18.165135    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81d98002-64a0-4881-b58a-5eba5848ff97-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m\" (UID: \"81d98002-64a0-4881-b58a-5eba5848ff97\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m"
Sep 15 20:19:18.165166 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:18.165167    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81d98002-64a0-4881-b58a-5eba5848ff97-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m\" (UID: \"81d98002-64a0-4881-b58a-5eba5848ff97\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m"
Sep 15 20:19:18.165354 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:18.165196    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81d98002-64a0-4881-b58a-5eba5848ff97-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m\" (UID: \"81d98002-64a0-4881-b58a-5eba5848ff97\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m"
Sep 15 20:19:18.165354 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:18.165265    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81d98002-64a0-4881-b58a-5eba5848ff97-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m\" (UID: \"81d98002-64a0-4881-b58a-5eba5848ff97\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m"
Sep 15 20:19:18.165354 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:18.165288    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81d98002-64a0-4881-b58a-5eba5848ff97-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m\" (UID: \"81d98002-64a0-4881-b58a-5eba5848ff97\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m"
Sep 15 20:19:18.165515 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:18.165422    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxgf9\" (UniqueName: \"kubernetes.io/projected/81d98002-64a0-4881-b58a-5eba5848ff97-kube-api-access-bxgf9\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m\" (UID: \"81d98002-64a0-4881-b58a-5eba5848ff97\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m"
Sep 15 20:19:18.165868 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:18.165837    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81d98002-64a0-4881-b58a-5eba5848ff97-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m\" (UID: \"81d98002-64a0-4881-b58a-5eba5848ff97\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m"
Sep 15 20:19:18.165868 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:18.165865    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81d98002-64a0-4881-b58a-5eba5848ff97-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m\" (UID: \"81d98002-64a0-4881-b58a-5eba5848ff97\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m"
Sep 15 20:19:18.166041 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:18.165907    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81d98002-64a0-4881-b58a-5eba5848ff97-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m\" (UID: \"81d98002-64a0-4881-b58a-5eba5848ff97\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m"
Sep 15 20:19:18.167423 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:18.167396    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81d98002-64a0-4881-b58a-5eba5848ff97-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m\" (UID: \"81d98002-64a0-4881-b58a-5eba5848ff97\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m"
Sep 15 20:19:18.167722 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:18.167703    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81d98002-64a0-4881-b58a-5eba5848ff97-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m\" (UID: \"81d98002-64a0-4881-b58a-5eba5848ff97\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m"
Sep 15 20:19:18.173755 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:18.173734    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxgf9\" (UniqueName: \"kubernetes.io/projected/81d98002-64a0-4881-b58a-5eba5848ff97-kube-api-access-bxgf9\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m\" (UID: \"81d98002-64a0-4881-b58a-5eba5848ff97\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m"
Sep 15 20:19:18.343289 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:18.343215    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m"
Sep 15 20:19:18.613148 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:18.613060    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m"]
Sep 15 20:19:18.615345 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:19:18.615304    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod81d98002_64a0_4881_b58a_5eba5848ff97.slice/crio-4b19896316c1bcdd535a61af02b6c741bbd6fff7e4dec8f22107631b23ffd257 WatchSource:0}: Error finding container 4b19896316c1bcdd535a61af02b6c741bbd6fff7e4dec8f22107631b23ffd257: Status 404 returned error can't find the container with id 4b19896316c1bcdd535a61af02b6c741bbd6fff7e4dec8f22107631b23ffd257
Sep 15 20:19:19.584944 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:19.584889    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m" event={"ID":"81d98002-64a0-4881-b58a-5eba5848ff97","Type":"ContainerStarted","Data":"326958103e3ffd47353209eefa60e7a38a0047043535aa8c905887a53f6f61a1"}
Sep 15 20:19:19.584944 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:19.584947    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m" event={"ID":"81d98002-64a0-4881-b58a-5eba5848ff97","Type":"ContainerStarted","Data":"4b19896316c1bcdd535a61af02b6c741bbd6fff7e4dec8f22107631b23ffd257"}
Sep 15 20:19:19.602491 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:19.602444    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m" podStartSLOduration=1.4051038710000001 podStartE2EDuration="1.602428859s" totalImagesPullingTime="197.324988ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:19:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:19:19.601002876 +0000 UTC m=+874.089467354" watchObservedRunningTime="2026-09-15 20:19:19.602428859 +0000 UTC m=+874.090893335"
Sep 15 20:19:27.322335 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:27.322302    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:19:28.344170 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:28.344136    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m"
Sep 15 20:19:28.344535 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:28.344184    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m"
Sep 15 20:19:28.356871 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:28.356850    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m"
Sep 15 20:19:28.635043 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:28.634965    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m"
Sep 15 20:19:45.166903 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:19:45.166817    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:20:58.396384 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:20:58.396351    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b469d9fb-c98sk"]
Sep 15 20:20:58.399616 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:20:58.399600    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b469d9fb-c98sk"
Sep 15 20:20:58.410230 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:20:58.410207    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b469d9fb-c98sk"]
Sep 15 20:20:58.421095 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:20:58.421073    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d5bq\" (UniqueName: \"kubernetes.io/projected/38e3235c-f4c7-4404-9470-a9cc0f6f763a-kube-api-access-8d5bq\") pod \"authorino-b469d9fb-c98sk\" (UID: \"38e3235c-f4c7-4404-9470-a9cc0f6f763a\") " pod="kuadrant-system/authorino-b469d9fb-c98sk"
Sep 15 20:20:58.421194 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:20:58.421111    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/38e3235c-f4c7-4404-9470-a9cc0f6f763a-tls-cert\") pod \"authorino-b469d9fb-c98sk\" (UID: \"38e3235c-f4c7-4404-9470-a9cc0f6f763a\") " pod="kuadrant-system/authorino-b469d9fb-c98sk"
Sep 15 20:20:58.521634 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:20:58.521610    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d5bq\" (UniqueName: \"kubernetes.io/projected/38e3235c-f4c7-4404-9470-a9cc0f6f763a-kube-api-access-8d5bq\") pod \"authorino-b469d9fb-c98sk\" (UID: \"38e3235c-f4c7-4404-9470-a9cc0f6f763a\") " pod="kuadrant-system/authorino-b469d9fb-c98sk"
Sep 15 20:20:58.521744 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:20:58.521649    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/38e3235c-f4c7-4404-9470-a9cc0f6f763a-tls-cert\") pod \"authorino-b469d9fb-c98sk\" (UID: \"38e3235c-f4c7-4404-9470-a9cc0f6f763a\") " pod="kuadrant-system/authorino-b469d9fb-c98sk"
Sep 15 20:20:58.524065 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:20:58.524047    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/38e3235c-f4c7-4404-9470-a9cc0f6f763a-tls-cert\") pod \"authorino-b469d9fb-c98sk\" (UID: \"38e3235c-f4c7-4404-9470-a9cc0f6f763a\") " pod="kuadrant-system/authorino-b469d9fb-c98sk"
Sep 15 20:20:58.529364 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:20:58.529339    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d5bq\" (UniqueName: \"kubernetes.io/projected/38e3235c-f4c7-4404-9470-a9cc0f6f763a-kube-api-access-8d5bq\") pod \"authorino-b469d9fb-c98sk\" (UID: \"38e3235c-f4c7-4404-9470-a9cc0f6f763a\") " pod="kuadrant-system/authorino-b469d9fb-c98sk"
Sep 15 20:20:58.708770 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:20:58.708751    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b469d9fb-c98sk"
Sep 15 20:20:58.980209 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:20:58.980185    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b469d9fb-c98sk"]
Sep 15 20:20:58.982306 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:20:58.982267    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod38e3235c_f4c7_4404_9470_a9cc0f6f763a.slice/crio-e5cfdad53de2b7798c9d8ea06775797d1671f371436c1a4b7b79fb332a7bea1d WatchSource:0}: Error finding container e5cfdad53de2b7798c9d8ea06775797d1671f371436c1a4b7b79fb332a7bea1d: Status 404 returned error can't find the container with id e5cfdad53de2b7798c9d8ea06775797d1671f371436c1a4b7b79fb332a7bea1d
Sep 15 20:20:58.983653 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:20:58.983636    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 20:20:59.045580 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:20:59.045552    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b469d9fb-c98sk" event={"ID":"38e3235c-f4c7-4404-9470-a9cc0f6f763a","Type":"ContainerStarted","Data":"e5cfdad53de2b7798c9d8ea06775797d1671f371436c1a4b7b79fb332a7bea1d"}
Sep 15 20:21:00.052654 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:00.052616    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b469d9fb-c98sk" event={"ID":"38e3235c-f4c7-4404-9470-a9cc0f6f763a","Type":"ContainerStarted","Data":"e75773cc148b433d07b7c147eb63086379c303922ca3366271c9806c36afeea6"}
Sep 15 20:21:00.073605 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:00.073551    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b469d9fb-c98sk" podStartSLOduration=1.414142795 podStartE2EDuration="2.073532523s" totalImagesPullingTime="659.389728ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:20:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:21:00.07224035 +0000 UTC m=+974.560704819" watchObservedRunningTime="2026-09-15 20:21:00.073532523 +0000 UTC m=+974.561996999"
Sep 15 20:21:00.102277 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:00.102242    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-84574cdcc-bq92v"]
Sep 15 20:21:00.102489 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:00.102462    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-84574cdcc-bq92v" podUID="99f49138-c843-40ae-a1cf-631c8d42ef42" containerName="authorino" containerID="cri-o://15b1ac9ed7134f61b2e414da905ab3519dce96fd734f8b6a3ea129c4020f3343" gracePeriod=30
Sep 15 20:21:00.367526 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:00.367498    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84574cdcc-bq92v"
Sep 15 20:21:00.435085 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:00.435054    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/99f49138-c843-40ae-a1cf-631c8d42ef42-tls-cert\") pod \"99f49138-c843-40ae-a1cf-631c8d42ef42\" (UID: \"99f49138-c843-40ae-a1cf-631c8d42ef42\") "
Sep 15 20:21:00.435264 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:00.435112    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6k6mw\" (UniqueName: \"kubernetes.io/projected/99f49138-c843-40ae-a1cf-631c8d42ef42-kube-api-access-6k6mw\") pod \"99f49138-c843-40ae-a1cf-631c8d42ef42\" (UID: \"99f49138-c843-40ae-a1cf-631c8d42ef42\") "
Sep 15 20:21:00.437254 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:00.437221    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99f49138-c843-40ae-a1cf-631c8d42ef42-kube-api-access-6k6mw" (OuterVolumeSpecName: "kube-api-access-6k6mw") pod "99f49138-c843-40ae-a1cf-631c8d42ef42" (UID: "99f49138-c843-40ae-a1cf-631c8d42ef42"). InnerVolumeSpecName "kube-api-access-6k6mw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:21:00.445303 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:00.445280    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99f49138-c843-40ae-a1cf-631c8d42ef42-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "99f49138-c843-40ae-a1cf-631c8d42ef42" (UID: "99f49138-c843-40ae-a1cf-631c8d42ef42"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:21:00.536076 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:00.536042    2576 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/99f49138-c843-40ae-a1cf-631c8d42ef42-tls-cert\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:21:00.536076 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:00.536073    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6k6mw\" (UniqueName: \"kubernetes.io/projected/99f49138-c843-40ae-a1cf-631c8d42ef42-kube-api-access-6k6mw\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:21:00.640049 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:00.639964    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:00.727253 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:00.727221    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:01.061538 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:01.061493    2576 generic.go:356] "Generic (PLEG): container finished" podID="99f49138-c843-40ae-a1cf-631c8d42ef42" containerID="15b1ac9ed7134f61b2e414da905ab3519dce96fd734f8b6a3ea129c4020f3343" exitCode=0
Sep 15 20:21:01.062024 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:01.061593    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84574cdcc-bq92v"
Sep 15 20:21:01.062024 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:01.061645    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84574cdcc-bq92v" event={"ID":"99f49138-c843-40ae-a1cf-631c8d42ef42","Type":"ContainerDied","Data":"15b1ac9ed7134f61b2e414da905ab3519dce96fd734f8b6a3ea129c4020f3343"}
Sep 15 20:21:01.062024 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:01.061679    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84574cdcc-bq92v" event={"ID":"99f49138-c843-40ae-a1cf-631c8d42ef42","Type":"ContainerDied","Data":"ac46caff0e08e4094a72bf4563810592cca7df4834a449f77e02ce10c7f007e9"}
Sep 15 20:21:01.062024 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:01.061720    2576 scope.go:117] "RemoveContainer" containerID="15b1ac9ed7134f61b2e414da905ab3519dce96fd734f8b6a3ea129c4020f3343"
Sep 15 20:21:01.074854 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:01.074825    2576 scope.go:117] "RemoveContainer" containerID="15b1ac9ed7134f61b2e414da905ab3519dce96fd734f8b6a3ea129c4020f3343"
Sep 15 20:21:01.076849 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:21:01.075175    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15b1ac9ed7134f61b2e414da905ab3519dce96fd734f8b6a3ea129c4020f3343\": container with ID starting with 15b1ac9ed7134f61b2e414da905ab3519dce96fd734f8b6a3ea129c4020f3343 not found: ID does not exist" containerID="15b1ac9ed7134f61b2e414da905ab3519dce96fd734f8b6a3ea129c4020f3343"
Sep 15 20:21:01.076849 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:01.075212    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15b1ac9ed7134f61b2e414da905ab3519dce96fd734f8b6a3ea129c4020f3343"} err="failed to get container status \"15b1ac9ed7134f61b2e414da905ab3519dce96fd734f8b6a3ea129c4020f3343\": rpc error: code = NotFound desc = could not find container \"15b1ac9ed7134f61b2e414da905ab3519dce96fd734f8b6a3ea129c4020f3343\": container with ID starting with 15b1ac9ed7134f61b2e414da905ab3519dce96fd734f8b6a3ea129c4020f3343 not found: ID does not exist"
Sep 15 20:21:01.093123 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:01.093087    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-84574cdcc-bq92v"]
Sep 15 20:21:01.100817 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:01.100793    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-84574cdcc-bq92v"]
Sep 15 20:21:01.907198 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:01.907165    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:02.019776 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:02.019741    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:02.165217 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:02.165136    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99f49138-c843-40ae-a1cf-631c8d42ef42" path="/var/lib/kubelet/pods/99f49138-c843-40ae-a1cf-631c8d42ef42/volumes"
Sep 15 20:21:19.949142 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:19.949107    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:20.062711 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:20.062678    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:20.166088 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:20.166055    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:20.364224 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:20.364190    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:20.464609 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:20.464577    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:20.535411 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:20.535375    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:20.603827 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:20.603798    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:20.704044 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:20.704019    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:21.631462 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:21.631415    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:21.732637 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:21.732603    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:22.139842 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:22.139804    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:23.860410 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:23.860359    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:23.963432 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:23.963383    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:24.060026 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:24.059994    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:24.277040 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:24.277005    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:24.380149 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:24.380115    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:24.480250 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:24.480220    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:25.087198 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:25.087166    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:25.229006 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:25.228971    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:25.529137 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:25.529108    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:25.749437 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:25.749407    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:25.841551 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:25.841464    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:25.842044 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:25.842024    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-7e3543-649dcfdd74-nm9vg"]
Sep 15 20:21:25.842440 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:25.842424    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99f49138-c843-40ae-a1cf-631c8d42ef42" containerName="authorino"
Sep 15 20:21:25.842514 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:25.842443    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="99f49138-c843-40ae-a1cf-631c8d42ef42" containerName="authorino"
Sep 15 20:21:25.842572 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:25.842520    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="99f49138-c843-40ae-a1cf-631c8d42ef42" containerName="authorino"
Sep 15 20:21:25.845455 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:25.845435    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7e3543-649dcfdd74-nm9vg"
Sep 15 20:21:25.847570 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:25.847556    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-7e3543\""
Sep 15 20:21:25.855495 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:25.855476    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-7e3543-649dcfdd74-nm9vg"]
Sep 15 20:21:25.882083 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:25.882058    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-7e3543-59dfc588fc-knvf7"]
Sep 15 20:21:25.885334 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:25.885313    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-7e3543-59dfc588fc-knvf7"
Sep 15 20:21:25.887517 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:25.887500    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-7e3543-dockercfg-ngv5j\""
Sep 15 20:21:25.895053 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:25.895031    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-7e3543-59dfc588fc-knvf7"]
Sep 15 20:21:25.956907 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:25.956874    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95418b12-f771-4e7c-836e-bb5e04eac166-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-7e3543-649dcfdd74-nm9vg\" (UID: \"95418b12-f771-4e7c-836e-bb5e04eac166\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7e3543-649dcfdd74-nm9vg"
Sep 15 20:21:25.957052 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:25.956952    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7912229-70e1-4ec7-8c05-7f756a8a9411-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-7e3543-59dfc588fc-knvf7\" (UID: \"a7912229-70e1-4ec7-8c05-7f756a8a9411\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-7e3543-59dfc588fc-knvf7"
Sep 15 20:21:25.957052 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:25.956989    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcpw2\" (UniqueName: \"kubernetes.io/projected/a7912229-70e1-4ec7-8c05-7f756a8a9411-kube-api-access-kcpw2\") pod \"payload-processing-e2e-worker-w0-7e3543-59dfc588fc-knvf7\" (UID: \"a7912229-70e1-4ec7-8c05-7f756a8a9411\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-7e3543-59dfc588fc-knvf7"
Sep 15 20:21:25.957052 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:25.957027    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5rnl\" (UniqueName: \"kubernetes.io/projected/95418b12-f771-4e7c-836e-bb5e04eac166-kube-api-access-s5rnl\") pod \"payload-pre-processing-e2e-worker-w0-7e3543-649dcfdd74-nm9vg\" (UID: \"95418b12-f771-4e7c-836e-bb5e04eac166\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7e3543-649dcfdd74-nm9vg"
Sep 15 20:21:26.057431 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:26.057403    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7912229-70e1-4ec7-8c05-7f756a8a9411-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-7e3543-59dfc588fc-knvf7\" (UID: \"a7912229-70e1-4ec7-8c05-7f756a8a9411\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-7e3543-59dfc588fc-knvf7"
Sep 15 20:21:26.057577 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:26.057445    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcpw2\" (UniqueName: \"kubernetes.io/projected/a7912229-70e1-4ec7-8c05-7f756a8a9411-kube-api-access-kcpw2\") pod \"payload-processing-e2e-worker-w0-7e3543-59dfc588fc-knvf7\" (UID: \"a7912229-70e1-4ec7-8c05-7f756a8a9411\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-7e3543-59dfc588fc-knvf7"
Sep 15 20:21:26.057577 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:26.057480    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5rnl\" (UniqueName: \"kubernetes.io/projected/95418b12-f771-4e7c-836e-bb5e04eac166-kube-api-access-s5rnl\") pod \"payload-pre-processing-e2e-worker-w0-7e3543-649dcfdd74-nm9vg\" (UID: \"95418b12-f771-4e7c-836e-bb5e04eac166\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7e3543-649dcfdd74-nm9vg"
Sep 15 20:21:26.057577 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:26.057505    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95418b12-f771-4e7c-836e-bb5e04eac166-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-7e3543-649dcfdd74-nm9vg\" (UID: \"95418b12-f771-4e7c-836e-bb5e04eac166\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7e3543-649dcfdd74-nm9vg"
Sep 15 20:21:26.058115 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:26.058094    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7912229-70e1-4ec7-8c05-7f756a8a9411-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-7e3543-59dfc588fc-knvf7\" (UID: \"a7912229-70e1-4ec7-8c05-7f756a8a9411\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-7e3543-59dfc588fc-knvf7"
Sep 15 20:21:26.058244 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:26.058227    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95418b12-f771-4e7c-836e-bb5e04eac166-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-7e3543-649dcfdd74-nm9vg\" (UID: \"95418b12-f771-4e7c-836e-bb5e04eac166\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7e3543-649dcfdd74-nm9vg"
Sep 15 20:21:26.066363 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:26.066343    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcpw2\" (UniqueName: \"kubernetes.io/projected/a7912229-70e1-4ec7-8c05-7f756a8a9411-kube-api-access-kcpw2\") pod \"payload-processing-e2e-worker-w0-7e3543-59dfc588fc-knvf7\" (UID: \"a7912229-70e1-4ec7-8c05-7f756a8a9411\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-7e3543-59dfc588fc-knvf7"
Sep 15 20:21:26.066462 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:26.066344    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5rnl\" (UniqueName: \"kubernetes.io/projected/95418b12-f771-4e7c-836e-bb5e04eac166-kube-api-access-s5rnl\") pod \"payload-pre-processing-e2e-worker-w0-7e3543-649dcfdd74-nm9vg\" (UID: \"95418b12-f771-4e7c-836e-bb5e04eac166\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7e3543-649dcfdd74-nm9vg"
Sep 15 20:21:26.156745 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:26.156674    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7e3543-649dcfdd74-nm9vg"
Sep 15 20:21:26.191492 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:26.191457    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:26.196191 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:26.196168    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-7e3543-59dfc588fc-knvf7"
Sep 15 20:21:26.259869 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:26.259825    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:26.417600 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:26.417525    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:26.438359 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:26.438338    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-7e3543-649dcfdd74-nm9vg"]
Sep 15 20:21:26.440399 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:21:26.440375    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95418b12_f771_4e7c_836e_bb5e04eac166.slice/crio-411d05c9814b3525b9646c66129efc77fed110febc97c07a1c2faa075720012d WatchSource:0}: Error finding container 411d05c9814b3525b9646c66129efc77fed110febc97c07a1c2faa075720012d: Status 404 returned error can't find the container with id 411d05c9814b3525b9646c66129efc77fed110febc97c07a1c2faa075720012d
Sep 15 20:21:26.488954 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:26.488914    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-7e3543-59dfc588fc-knvf7"]
Sep 15 20:21:26.490218 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:21:26.490196    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7912229_70e1_4ec7_8c05_7f756a8a9411.slice/crio-cad2d2bb330d278ed248aa56e17d5cc12428dfb021649aff67ca8d26736863b4 WatchSource:0}: Error finding container cad2d2bb330d278ed248aa56e17d5cc12428dfb021649aff67ca8d26736863b4: Status 404 returned error can't find the container with id cad2d2bb330d278ed248aa56e17d5cc12428dfb021649aff67ca8d26736863b4
Sep 15 20:21:26.792655 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:26.792629    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:26.899149 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:26.899118    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-01ea3b-7c64f65746-h5cxj"]
Sep 15 20:21:26.902656 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:26.902636    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-01ea3b-7c64f65746-h5cxj"
Sep 15 20:21:26.905183 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:26.905162    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-01ea3b-dockercfg-cl7kb\""
Sep 15 20:21:26.905183 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:26.905184    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-01ea3b\""
Sep 15 20:21:26.912755 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:26.912728    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-01ea3b-7c64f65746-h5cxj"]
Sep 15 20:21:26.915292 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:26.915274    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-01ea3b-5cfd8d6bdf-ln7n4"]
Sep 15 20:21:26.918574 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:26.918543    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-01ea3b-5cfd8d6bdf-ln7n4"
Sep 15 20:21:26.930658 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:26.930622    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-01ea3b-5cfd8d6bdf-ln7n4"]
Sep 15 20:21:26.975129 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:26.975074    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:27.066409 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:27.066341    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdv2v\" (UniqueName: \"kubernetes.io/projected/4fad2e57-3655-4526-a43e-8d57820aba63-kube-api-access-mdv2v\") pod \"payload-processing-e2e-inf-a-01ea3b-5cfd8d6bdf-ln7n4\" (UID: \"4fad2e57-3655-4526-a43e-8d57820aba63\") " pod="openshift-ingress/payload-processing-e2e-inf-a-01ea3b-5cfd8d6bdf-ln7n4"
Sep 15 20:21:27.066518 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:27.066406    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fad2e57-3655-4526-a43e-8d57820aba63-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-01ea3b-5cfd8d6bdf-ln7n4\" (UID: \"4fad2e57-3655-4526-a43e-8d57820aba63\") " pod="openshift-ingress/payload-processing-e2e-inf-a-01ea3b-5cfd8d6bdf-ln7n4"
Sep 15 20:21:27.066518 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:27.066488    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a5673a7-326b-441f-9fd6-2a3560e16199-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-01ea3b-7c64f65746-h5cxj\" (UID: \"5a5673a7-326b-441f-9fd6-2a3560e16199\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-01ea3b-7c64f65746-h5cxj"
Sep 15 20:21:27.066625 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:27.066589    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28ws7\" (UniqueName: \"kubernetes.io/projected/5a5673a7-326b-441f-9fd6-2a3560e16199-kube-api-access-28ws7\") pod \"payload-pre-processing-e2e-inf-a-01ea3b-7c64f65746-h5cxj\" (UID: \"5a5673a7-326b-441f-9fd6-2a3560e16199\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-01ea3b-7c64f65746-h5cxj"
Sep 15 20:21:27.167672 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:27.167640    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a5673a7-326b-441f-9fd6-2a3560e16199-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-01ea3b-7c64f65746-h5cxj\" (UID: \"5a5673a7-326b-441f-9fd6-2a3560e16199\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-01ea3b-7c64f65746-h5cxj"
Sep 15 20:21:27.168088 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:27.167707    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28ws7\" (UniqueName: \"kubernetes.io/projected/5a5673a7-326b-441f-9fd6-2a3560e16199-kube-api-access-28ws7\") pod \"payload-pre-processing-e2e-inf-a-01ea3b-7c64f65746-h5cxj\" (UID: \"5a5673a7-326b-441f-9fd6-2a3560e16199\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-01ea3b-7c64f65746-h5cxj"
Sep 15 20:21:27.168088 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:27.167747    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdv2v\" (UniqueName: \"kubernetes.io/projected/4fad2e57-3655-4526-a43e-8d57820aba63-kube-api-access-mdv2v\") pod \"payload-processing-e2e-inf-a-01ea3b-5cfd8d6bdf-ln7n4\" (UID: \"4fad2e57-3655-4526-a43e-8d57820aba63\") " pod="openshift-ingress/payload-processing-e2e-inf-a-01ea3b-5cfd8d6bdf-ln7n4"
Sep 15 20:21:27.168088 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:27.167850    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fad2e57-3655-4526-a43e-8d57820aba63-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-01ea3b-5cfd8d6bdf-ln7n4\" (UID: \"4fad2e57-3655-4526-a43e-8d57820aba63\") " pod="openshift-ingress/payload-processing-e2e-inf-a-01ea3b-5cfd8d6bdf-ln7n4"
Sep 15 20:21:27.168473 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:27.168453    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a5673a7-326b-441f-9fd6-2a3560e16199-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-01ea3b-7c64f65746-h5cxj\" (UID: \"5a5673a7-326b-441f-9fd6-2a3560e16199\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-01ea3b-7c64f65746-h5cxj"
Sep 15 20:21:27.168679 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:27.168659    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fad2e57-3655-4526-a43e-8d57820aba63-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-01ea3b-5cfd8d6bdf-ln7n4\" (UID: \"4fad2e57-3655-4526-a43e-8d57820aba63\") " pod="openshift-ingress/payload-processing-e2e-inf-a-01ea3b-5cfd8d6bdf-ln7n4"
Sep 15 20:21:27.177597 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:27.177571    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdv2v\" (UniqueName: \"kubernetes.io/projected/4fad2e57-3655-4526-a43e-8d57820aba63-kube-api-access-mdv2v\") pod \"payload-processing-e2e-inf-a-01ea3b-5cfd8d6bdf-ln7n4\" (UID: \"4fad2e57-3655-4526-a43e-8d57820aba63\") " pod="openshift-ingress/payload-processing-e2e-inf-a-01ea3b-5cfd8d6bdf-ln7n4"
Sep 15 20:21:27.177706 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:27.177615    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28ws7\" (UniqueName: \"kubernetes.io/projected/5a5673a7-326b-441f-9fd6-2a3560e16199-kube-api-access-28ws7\") pod \"payload-pre-processing-e2e-inf-a-01ea3b-7c64f65746-h5cxj\" (UID: \"5a5673a7-326b-441f-9fd6-2a3560e16199\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-01ea3b-7c64f65746-h5cxj"
Sep 15 20:21:27.188827 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:27.188798    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-7e3543-59dfc588fc-knvf7" event={"ID":"a7912229-70e1-4ec7-8c05-7f756a8a9411","Type":"ContainerStarted","Data":"ba541365fec731519309143327d4f1fdddc9e8b495f5c18ab18adfc555882851"}
Sep 15 20:21:27.188973 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:27.188856    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-7e3543-59dfc588fc-knvf7" event={"ID":"a7912229-70e1-4ec7-8c05-7f756a8a9411","Type":"ContainerStarted","Data":"cad2d2bb330d278ed248aa56e17d5cc12428dfb021649aff67ca8d26736863b4"}
Sep 15 20:21:27.188973 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:27.188886    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-7e3543-59dfc588fc-knvf7"
Sep 15 20:21:27.190440 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:27.190419    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7e3543-649dcfdd74-nm9vg" event={"ID":"95418b12-f771-4e7c-836e-bb5e04eac166","Type":"ContainerStarted","Data":"075039eefb0f02375bfa33c7c5cb9bcb15698e7609a646a41a089c82b12d21e9"}
Sep 15 20:21:27.190542 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:27.190455    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7e3543-649dcfdd74-nm9vg" event={"ID":"95418b12-f771-4e7c-836e-bb5e04eac166","Type":"ContainerStarted","Data":"411d05c9814b3525b9646c66129efc77fed110febc97c07a1c2faa075720012d"}
Sep 15 20:21:27.190542 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:27.190526    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7e3543-649dcfdd74-nm9vg"
Sep 15 20:21:27.210547 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:27.210487    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-7e3543-59dfc588fc-knvf7" podStartSLOduration=2.210474175 podStartE2EDuration="2.210474175s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:21:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:21:27.208537829 +0000 UTC m=+1001.697002317" watchObservedRunningTime="2026-09-15 20:21:27.210474175 +0000 UTC m=+1001.698938651"
Sep 15 20:21:27.215218 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:27.215199    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-01ea3b-7c64f65746-h5cxj"
Sep 15 20:21:27.218642 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:27.218617    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:27.228407 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:27.228356    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7e3543-649dcfdd74-nm9vg" podStartSLOduration=2.228338292 podStartE2EDuration="2.228338292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:21:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:21:27.226100079 +0000 UTC m=+1001.714564554" watchObservedRunningTime="2026-09-15 20:21:27.228338292 +0000 UTC m=+1001.716802769"
Sep 15 20:21:27.235316 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:27.235295    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-01ea3b-5cfd8d6bdf-ln7n4"
Sep 15 20:21:27.318321 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:27.318229    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:27.419985 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:27.419948    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:27.504836 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:27.504788    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-01ea3b-7c64f65746-h5cxj"]
Sep 15 20:21:27.509151 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:21:27.509112    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a5673a7_326b_441f_9fd6_2a3560e16199.slice/crio-c5c8a273d3a2683ee6a580e3662f85b571ec993c9f5d7b79cf5431570f9e82dc WatchSource:0}: Error finding container c5c8a273d3a2683ee6a580e3662f85b571ec993c9f5d7b79cf5431570f9e82dc: Status 404 returned error can't find the container with id c5c8a273d3a2683ee6a580e3662f85b571ec993c9f5d7b79cf5431570f9e82dc
Sep 15 20:21:27.537205 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:27.536769    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-01ea3b-5cfd8d6bdf-ln7n4"]
Sep 15 20:21:27.542454 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:21:27.542108    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fad2e57_3655_4526_a43e_8d57820aba63.slice/crio-70edb8568a11fb44b9daf820686edc7540f7f3bc21a5977cf1362d5859df902d WatchSource:0}: Error finding container 70edb8568a11fb44b9daf820686edc7540f7f3bc21a5977cf1362d5859df902d: Status 404 returned error can't find the container with id 70edb8568a11fb44b9daf820686edc7540f7f3bc21a5977cf1362d5859df902d
Sep 15 20:21:27.548465 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:27.548441    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:27.893682 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:27.893589    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:28.043347 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.043312    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-725e67-667855bfcd-cgzbp"]
Sep 15 20:21:28.047128 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.047102    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-725e67-667855bfcd-cgzbp"
Sep 15 20:21:28.049946 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.049907    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-725e67-667c64b48c-kz47q"]
Sep 15 20:21:28.050065 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.050040    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-725e67\""
Sep 15 20:21:28.054913 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.054895    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-725e67-667c64b48c-kz47q"
Sep 15 20:21:28.062364 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.062278    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:28.079516 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.079094    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-725e67-667855bfcd-cgzbp"]
Sep 15 20:21:28.092633 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.092600    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-725e67-667c64b48c-kz47q"]
Sep 15 20:21:28.177365 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.177279    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wv5bn\" (UniqueName: \"kubernetes.io/projected/fe36ea96-bfb6-4fd1-a260-a27872453328-kube-api-access-wv5bn\") pod \"payload-processing-e2e-worker-w4-725e67-667c64b48c-kz47q\" (UID: \"fe36ea96-bfb6-4fd1-a260-a27872453328\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-725e67-667c64b48c-kz47q"
Sep 15 20:21:28.177365 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.177323    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw722\" (UniqueName: \"kubernetes.io/projected/53826b73-d03f-4af1-a2aa-33825571d570-kube-api-access-xw722\") pod \"payload-pre-processing-e2e-worker-w4-725e67-667855bfcd-cgzbp\" (UID: \"53826b73-d03f-4af1-a2aa-33825571d570\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-725e67-667855bfcd-cgzbp"
Sep 15 20:21:28.177859 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.177472    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe36ea96-bfb6-4fd1-a260-a27872453328-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-725e67-667c64b48c-kz47q\" (UID: \"fe36ea96-bfb6-4fd1-a260-a27872453328\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-725e67-667c64b48c-kz47q"
Sep 15 20:21:28.177859 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.177521    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53826b73-d03f-4af1-a2aa-33825571d570-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-725e67-667855bfcd-cgzbp\" (UID: \"53826b73-d03f-4af1-a2aa-33825571d570\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-725e67-667855bfcd-cgzbp"
Sep 15 20:21:28.189107 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.189078    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:28.197665 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.197636    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-01ea3b-7c64f65746-h5cxj" event={"ID":"5a5673a7-326b-441f-9fd6-2a3560e16199","Type":"ContainerStarted","Data":"99d76e9629e25b0c841cf64dca7f97fd2b30045fa56d40f50aaa53cf14872600"}
Sep 15 20:21:28.197771 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.197687    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-01ea3b-7c64f65746-h5cxj" event={"ID":"5a5673a7-326b-441f-9fd6-2a3560e16199","Type":"ContainerStarted","Data":"c5c8a273d3a2683ee6a580e3662f85b571ec993c9f5d7b79cf5431570f9e82dc"}
Sep 15 20:21:28.197771 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.197717    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-01ea3b-7c64f65746-h5cxj"
Sep 15 20:21:28.199437 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.199413    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-01ea3b-5cfd8d6bdf-ln7n4" event={"ID":"4fad2e57-3655-4526-a43e-8d57820aba63","Type":"ContainerStarted","Data":"a1a06a9254dd55886965af730907ce8cf7dcb26201f500bde9676797d3af2a70"}
Sep 15 20:21:28.199554 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.199493    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-01ea3b-5cfd8d6bdf-ln7n4" event={"ID":"4fad2e57-3655-4526-a43e-8d57820aba63","Type":"ContainerStarted","Data":"70edb8568a11fb44b9daf820686edc7540f7f3bc21a5977cf1362d5859df902d"}
Sep 15 20:21:28.199740 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.199724    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-01ea3b-5cfd8d6bdf-ln7n4"
Sep 15 20:21:28.229212 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.229163    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-01ea3b-7c64f65746-h5cxj" podStartSLOduration=2.229147435 podStartE2EDuration="2.229147435s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:21:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:21:28.225285481 +0000 UTC m=+1002.713749957" watchObservedRunningTime="2026-09-15 20:21:28.229147435 +0000 UTC m=+1002.717611915"
Sep 15 20:21:28.252653 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.252600    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-01ea3b-5cfd8d6bdf-ln7n4" podStartSLOduration=2.252581479 podStartE2EDuration="2.252581479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:21:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:21:28.250752406 +0000 UTC m=+1002.739216908" watchObservedRunningTime="2026-09-15 20:21:28.252581479 +0000 UTC m=+1002.741045955"
Sep 15 20:21:28.279037 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.279011    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe36ea96-bfb6-4fd1-a260-a27872453328-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-725e67-667c64b48c-kz47q\" (UID: \"fe36ea96-bfb6-4fd1-a260-a27872453328\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-725e67-667c64b48c-kz47q"
Sep 15 20:21:28.279160 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.279084    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53826b73-d03f-4af1-a2aa-33825571d570-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-725e67-667855bfcd-cgzbp\" (UID: \"53826b73-d03f-4af1-a2aa-33825571d570\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-725e67-667855bfcd-cgzbp"
Sep 15 20:21:28.279226 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.279165    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wv5bn\" (UniqueName: \"kubernetes.io/projected/fe36ea96-bfb6-4fd1-a260-a27872453328-kube-api-access-wv5bn\") pod \"payload-processing-e2e-worker-w4-725e67-667c64b48c-kz47q\" (UID: \"fe36ea96-bfb6-4fd1-a260-a27872453328\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-725e67-667c64b48c-kz47q"
Sep 15 20:21:28.279226 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.279210    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw722\" (UniqueName: \"kubernetes.io/projected/53826b73-d03f-4af1-a2aa-33825571d570-kube-api-access-xw722\") pod \"payload-pre-processing-e2e-worker-w4-725e67-667855bfcd-cgzbp\" (UID: \"53826b73-d03f-4af1-a2aa-33825571d570\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-725e67-667855bfcd-cgzbp"
Sep 15 20:21:28.279874 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.279848    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe36ea96-bfb6-4fd1-a260-a27872453328-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-725e67-667c64b48c-kz47q\" (UID: \"fe36ea96-bfb6-4fd1-a260-a27872453328\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-725e67-667c64b48c-kz47q"
Sep 15 20:21:28.280005 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.279952    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53826b73-d03f-4af1-a2aa-33825571d570-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-725e67-667855bfcd-cgzbp\" (UID: \"53826b73-d03f-4af1-a2aa-33825571d570\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-725e67-667855bfcd-cgzbp"
Sep 15 20:21:28.288324 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.288302    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw722\" (UniqueName: \"kubernetes.io/projected/53826b73-d03f-4af1-a2aa-33825571d570-kube-api-access-xw722\") pod \"payload-pre-processing-e2e-worker-w4-725e67-667855bfcd-cgzbp\" (UID: \"53826b73-d03f-4af1-a2aa-33825571d570\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-725e67-667855bfcd-cgzbp"
Sep 15 20:21:28.288494 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.288474    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wv5bn\" (UniqueName: \"kubernetes.io/projected/fe36ea96-bfb6-4fd1-a260-a27872453328-kube-api-access-wv5bn\") pod \"payload-processing-e2e-worker-w4-725e67-667c64b48c-kz47q\" (UID: \"fe36ea96-bfb6-4fd1-a260-a27872453328\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-725e67-667c64b48c-kz47q"
Sep 15 20:21:28.358712 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.358687    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-725e67-667855bfcd-cgzbp"
Sep 15 20:21:28.393760 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.393733    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-725e67-667c64b48c-kz47q"
Sep 15 20:21:28.440629 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.440510    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:28.540287 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.540258    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:28.636633 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.636604    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-725e67-667855bfcd-cgzbp"]
Sep 15 20:21:28.639524 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:21:28.639498    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53826b73_d03f_4af1_a2aa_33825571d570.slice/crio-845b51d7582392f0b6467029e2dc8dce278fed8d707bed086ca2ee6c8d4a3641 WatchSource:0}: Error finding container 845b51d7582392f0b6467029e2dc8dce278fed8d707bed086ca2ee6c8d4a3641: Status 404 returned error can't find the container with id 845b51d7582392f0b6467029e2dc8dce278fed8d707bed086ca2ee6c8d4a3641
Sep 15 20:21:28.675547 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:28.675521    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-725e67-667c64b48c-kz47q"]
Sep 15 20:21:28.677077 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:21:28.677047    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe36ea96_bfb6_4fd1_a260_a27872453328.slice/crio-cc6f3b14fe78ae9c8511c25d07cba9b4033b26b8ee7a2bc9f4258c889e1a33f2 WatchSource:0}: Error finding container cc6f3b14fe78ae9c8511c25d07cba9b4033b26b8ee7a2bc9f4258c889e1a33f2: Status 404 returned error can't find the container with id cc6f3b14fe78ae9c8511c25d07cba9b4033b26b8ee7a2bc9f4258c889e1a33f2
Sep 15 20:21:29.206152 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:29.206111    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-725e67-667855bfcd-cgzbp" event={"ID":"53826b73-d03f-4af1-a2aa-33825571d570","Type":"ContainerStarted","Data":"694e6158c90dd5d733fdf973f79b35923c49673644ee32e88b5a78e2d18adeb8"}
Sep 15 20:21:29.206582 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:29.206163    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-725e67-667855bfcd-cgzbp" event={"ID":"53826b73-d03f-4af1-a2aa-33825571d570","Type":"ContainerStarted","Data":"845b51d7582392f0b6467029e2dc8dce278fed8d707bed086ca2ee6c8d4a3641"}
Sep 15 20:21:29.206582 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:29.206196    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-725e67-667855bfcd-cgzbp"
Sep 15 20:21:29.207730 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:29.207699    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-725e67-667c64b48c-kz47q" event={"ID":"fe36ea96-bfb6-4fd1-a260-a27872453328","Type":"ContainerStarted","Data":"b6ca48352debbd5b57ef5427bb047c0091fb29d9c150c4bd232265c1ae11f3e2"}
Sep 15 20:21:29.207854 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:29.207747    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-725e67-667c64b48c-kz47q" event={"ID":"fe36ea96-bfb6-4fd1-a260-a27872453328","Type":"ContainerStarted","Data":"cc6f3b14fe78ae9c8511c25d07cba9b4033b26b8ee7a2bc9f4258c889e1a33f2"}
Sep 15 20:21:29.208044 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:29.208024    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-725e67-667c64b48c-kz47q"
Sep 15 20:21:29.225115 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:29.225072    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-725e67-667855bfcd-cgzbp" podStartSLOduration=1.225059928 podStartE2EDuration="1.225059928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:21:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:21:29.22278418 +0000 UTC m=+1003.711248656" watchObservedRunningTime="2026-09-15 20:21:29.225059928 +0000 UTC m=+1003.713524403"
Sep 15 20:21:29.272034 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:29.272006    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:29.886911 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:29.886876    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:29.997114 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:29.997079    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:30.135755 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:30.135722    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:30.391134 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:30.391095    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:30.476502 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:30.476461    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:30.730019 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:30.729981    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:30.874132 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:30.874103    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:30.963732 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:30.963694    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:31.464863 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:31.464826    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:32.378144 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:32.378085    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:33.101854 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:33.101815    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:33.394005 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:33.393839    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:33.486350 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:33.486310    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:34.170150 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:34.170107    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:34.913638 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:34.913584    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:35.017654 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:35.017622    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:35.119866 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:35.119826    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:35.210871 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:35.210838    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:35.587415 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:35.587341    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:35.749019 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:35.748988    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:35.801889 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:35.801828    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-725e67-667c64b48c-kz47q" podStartSLOduration=7.801810636 podStartE2EDuration="7.801810636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:21:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:21:29.245059311 +0000 UTC m=+1003.733523787" watchObservedRunningTime="2026-09-15 20:21:35.801810636 +0000 UTC m=+1010.290275111"
Sep 15 20:21:35.803599 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:35.803572    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-30d5aa-79f75c48cd-fx84v"]
Sep 15 20:21:35.808350 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:35.808330    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-30d5aa-79f75c48cd-fx84v"
Sep 15 20:21:35.810462 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:35.810445    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-30d5aa\""
Sep 15 20:21:35.815435 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:35.815414    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-30d5aa-79f75c48cd-fx84v"]
Sep 15 20:21:35.818697 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:35.818675    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:35.845204 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:35.845149    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-30d5aa-577f5b65d9-hbrbf"]
Sep 15 20:21:35.848681 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:35.848667    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-30d5aa-577f5b65d9-hbrbf"
Sep 15 20:21:35.850389 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:35.850362    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7tt8\" (UniqueName: \"kubernetes.io/projected/49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672-kube-api-access-x7tt8\") pod \"payload-processing-e2e-autores-30d5aa-577f5b65d9-hbrbf\" (UID: \"49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672\") " pod="openshift-ingress/payload-processing-e2e-autores-30d5aa-577f5b65d9-hbrbf"
Sep 15 20:21:35.850488 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:35.850394    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpwdb\" (UniqueName: \"kubernetes.io/projected/0d95abd6-9b65-4712-8979-449ff240db58-kube-api-access-cpwdb\") pod \"payload-pre-processing-e2e-autores-30d5aa-79f75c48cd-fx84v\" (UID: \"0d95abd6-9b65-4712-8979-449ff240db58\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-30d5aa-79f75c48cd-fx84v"
Sep 15 20:21:35.850488 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:35.850420    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672-plugins-config-volume\") pod \"payload-processing-e2e-autores-30d5aa-577f5b65d9-hbrbf\" (UID: \"49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672\") " pod="openshift-ingress/payload-processing-e2e-autores-30d5aa-577f5b65d9-hbrbf"
Sep 15 20:21:35.850488 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:35.850448    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d95abd6-9b65-4712-8979-449ff240db58-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-30d5aa-79f75c48cd-fx84v\" (UID: \"0d95abd6-9b65-4712-8979-449ff240db58\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-30d5aa-79f75c48cd-fx84v"
Sep 15 20:21:35.852031 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:35.852017    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-30d5aa-dockercfg-hhbn4\""
Sep 15 20:21:35.864190 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:35.864167    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-30d5aa-577f5b65d9-hbrbf"]
Sep 15 20:21:35.951723 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:35.951689    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672-plugins-config-volume\") pod \"payload-processing-e2e-autores-30d5aa-577f5b65d9-hbrbf\" (UID: \"49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672\") " pod="openshift-ingress/payload-processing-e2e-autores-30d5aa-577f5b65d9-hbrbf"
Sep 15 20:21:35.951723 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:35.951727    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d95abd6-9b65-4712-8979-449ff240db58-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-30d5aa-79f75c48cd-fx84v\" (UID: \"0d95abd6-9b65-4712-8979-449ff240db58\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-30d5aa-79f75c48cd-fx84v"
Sep 15 20:21:35.951951 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:35.951804    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7tt8\" (UniqueName: \"kubernetes.io/projected/49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672-kube-api-access-x7tt8\") pod \"payload-processing-e2e-autores-30d5aa-577f5b65d9-hbrbf\" (UID: \"49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672\") " pod="openshift-ingress/payload-processing-e2e-autores-30d5aa-577f5b65d9-hbrbf"
Sep 15 20:21:35.951951 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:35.951830    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpwdb\" (UniqueName: \"kubernetes.io/projected/0d95abd6-9b65-4712-8979-449ff240db58-kube-api-access-cpwdb\") pod \"payload-pre-processing-e2e-autores-30d5aa-79f75c48cd-fx84v\" (UID: \"0d95abd6-9b65-4712-8979-449ff240db58\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-30d5aa-79f75c48cd-fx84v"
Sep 15 20:21:35.952376 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:35.952357    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672-plugins-config-volume\") pod \"payload-processing-e2e-autores-30d5aa-577f5b65d9-hbrbf\" (UID: \"49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672\") " pod="openshift-ingress/payload-processing-e2e-autores-30d5aa-577f5b65d9-hbrbf"
Sep 15 20:21:35.952493 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:35.952477    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d95abd6-9b65-4712-8979-449ff240db58-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-30d5aa-79f75c48cd-fx84v\" (UID: \"0d95abd6-9b65-4712-8979-449ff240db58\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-30d5aa-79f75c48cd-fx84v"
Sep 15 20:21:35.962284 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:35.962263    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpwdb\" (UniqueName: \"kubernetes.io/projected/0d95abd6-9b65-4712-8979-449ff240db58-kube-api-access-cpwdb\") pod \"payload-pre-processing-e2e-autores-30d5aa-79f75c48cd-fx84v\" (UID: \"0d95abd6-9b65-4712-8979-449ff240db58\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-30d5aa-79f75c48cd-fx84v"
Sep 15 20:21:35.962486 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:35.962464    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7tt8\" (UniqueName: \"kubernetes.io/projected/49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672-kube-api-access-x7tt8\") pod \"payload-processing-e2e-autores-30d5aa-577f5b65d9-hbrbf\" (UID: \"49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672\") " pod="openshift-ingress/payload-processing-e2e-autores-30d5aa-577f5b65d9-hbrbf"
Sep 15 20:21:36.120697 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:36.120623    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-30d5aa-79f75c48cd-fx84v"
Sep 15 20:21:36.142306 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:36.142277    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:36.158828 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:36.158803    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-30d5aa-577f5b65d9-hbrbf"
Sep 15 20:21:36.400856 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:36.400831    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-30d5aa-79f75c48cd-fx84v"]
Sep 15 20:21:36.402705 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:21:36.402681    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d95abd6_9b65_4712_8979_449ff240db58.slice/crio-822c86afc923a14501e56fdd0cd247041977887385dec6fd5191cb866d781cdf WatchSource:0}: Error finding container 822c86afc923a14501e56fdd0cd247041977887385dec6fd5191cb866d781cdf: Status 404 returned error can't find the container with id 822c86afc923a14501e56fdd0cd247041977887385dec6fd5191cb866d781cdf
Sep 15 20:21:36.452295 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:36.452272    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-30d5aa-577f5b65d9-hbrbf"]
Sep 15 20:21:36.453616 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:21:36.453591    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49d7ffc8_3eae_4c47_ae6e_bfdbf6d66672.slice/crio-319c6643ec3f4c47f159f5835f4bce5f3c45cb1c8d1b2f2bb999603ede4d22a7 WatchSource:0}: Error finding container 319c6643ec3f4c47f159f5835f4bce5f3c45cb1c8d1b2f2bb999603ede4d22a7: Status 404 returned error can't find the container with id 319c6643ec3f4c47f159f5835f4bce5f3c45cb1c8d1b2f2bb999603ede4d22a7
Sep 15 20:21:37.139421 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:37.139384    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:37.246768 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:37.246727    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:37.253441 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:37.253405    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-30d5aa-79f75c48cd-fx84v" event={"ID":"0d95abd6-9b65-4712-8979-449ff240db58","Type":"ContainerStarted","Data":"c68010ec58a3ce5d7cd65d5b96df615e917ada741a5a8de851cc3404ed34894c"}
Sep 15 20:21:37.253596 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:37.253467    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-30d5aa-79f75c48cd-fx84v" event={"ID":"0d95abd6-9b65-4712-8979-449ff240db58","Type":"ContainerStarted","Data":"822c86afc923a14501e56fdd0cd247041977887385dec6fd5191cb866d781cdf"}
Sep 15 20:21:37.253691 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:37.253675    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-30d5aa-79f75c48cd-fx84v"
Sep 15 20:21:37.255789 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:37.255765    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-30d5aa-577f5b65d9-hbrbf" event={"ID":"49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672","Type":"ContainerStarted","Data":"8a6cc6fe4423bab6cee2a094ffb7d9ec9b0b69a4be6b726dfc70e2fc57a526dd"}
Sep 15 20:21:37.255997 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:37.255973    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-30d5aa-577f5b65d9-hbrbf" event={"ID":"49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672","Type":"ContainerStarted","Data":"319c6643ec3f4c47f159f5835f4bce5f3c45cb1c8d1b2f2bb999603ede4d22a7"}
Sep 15 20:21:37.256135 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:37.256011    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-30d5aa-577f5b65d9-hbrbf"
Sep 15 20:21:37.273202 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:37.273147    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-30d5aa-79f75c48cd-fx84v" podStartSLOduration=2.273126793 podStartE2EDuration="2.273126793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:21:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:21:37.270553363 +0000 UTC m=+1011.759017841" watchObservedRunningTime="2026-09-15 20:21:37.273126793 +0000 UTC m=+1011.761591270"
Sep 15 20:21:37.290862 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:37.290813    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-30d5aa-577f5b65d9-hbrbf" podStartSLOduration=2.290796176 podStartE2EDuration="2.290796176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:21:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:21:37.290160673 +0000 UTC m=+1011.778625148" watchObservedRunningTime="2026-09-15 20:21:37.290796176 +0000 UTC m=+1011.779260653"
Sep 15 20:21:37.544937 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:37.544891    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:37.740780 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:37.740747    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:37.840792 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:37.840717    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:38.201506 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:38.201430    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-7e3543-59dfc588fc-knvf7"
Sep 15 20:21:38.201506 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:38.201491    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7e3543-649dcfdd74-nm9vg"
Sep 15 20:21:38.363937 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:38.363896    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:38.632383 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:38.632355    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:39.076719 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:39.076681    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:39.168032 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:39.168004    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:39.210069 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:39.210043    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-01ea3b-7c64f65746-h5cxj"
Sep 15 20:21:39.269867 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:39.269834    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:40.215477 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:40.215452    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-725e67-667c64b48c-kz47q"
Sep 15 20:21:40.215909 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:40.215517    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-725e67-667855bfcd-cgzbp"
Sep 15 20:21:40.215909 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:40.215803    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-01ea3b-5cfd8d6bdf-ln7n4"
Sep 15 20:21:40.312039 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:40.312010    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:40.894134 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:40.894099    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:41.063798 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:41.063766    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:41.206018 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:41.200614    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:41.387968 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:41.387915    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:41.570947 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:41.570835    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:42.213342 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:42.213308    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:42.323037 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:42.323004    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:42.423370 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:42.423341    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:42.522681 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:42.522613    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:42.624470 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:42.624439    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:43.517613 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:43.517580    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:43.785121 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:43.785045    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:43.995042 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:43.994991    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:44.096370 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:44.096298    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:44.217848 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:44.217821    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:44.371004 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:44.370906    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:44.444502 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:44.444467    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:44.548799 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:44.548769    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:44.689173 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:44.689084    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:44.785973 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:44.785947    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:45.677227 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:45.677194    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:45.777145 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:45.777113    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:46.573626 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:46.573577    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:47.368034 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:47.368000    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:47.970698 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:47.970664    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:48.062799 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:48.062764    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:48.265760 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:48.265686    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-30d5aa-79f75c48cd-fx84v"
Sep 15 20:21:48.265897 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:48.265761    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-30d5aa-577f5b65d9-hbrbf"
Sep 15 20:21:48.310677 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:48.310644    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:49.249581 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:49.249546    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:49.341795 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:49.341760    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:49.523916 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:49.523813    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:49.753554 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:49.753515    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:49.853973 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:49.853847    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:49.874854 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:49.874824    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-39c7b0-56db6464b6-6tgv8"]
Sep 15 20:21:49.880098 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:49.880076    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39c7b0-56db6464b6-6tgv8"
Sep 15 20:21:49.882738 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:49.882715    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-39c7b0\""
Sep 15 20:21:49.908596 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:49.908568    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-39c7b0-56db6464b6-6tgv8"]
Sep 15 20:21:49.984346 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:49.984311    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5x6dz\" (UniqueName: \"kubernetes.io/projected/90ad8ae4-67bf-49ae-bfcb-19d551508307-kube-api-access-5x6dz\") pod \"payload-pre-processing-e2e-inf-b-39c7b0-56db6464b6-6tgv8\" (UID: \"90ad8ae4-67bf-49ae-bfcb-19d551508307\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39c7b0-56db6464b6-6tgv8"
Sep 15 20:21:49.984521 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:49.984362    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90ad8ae4-67bf-49ae-bfcb-19d551508307-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-39c7b0-56db6464b6-6tgv8\" (UID: \"90ad8ae4-67bf-49ae-bfcb-19d551508307\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39c7b0-56db6464b6-6tgv8"
Sep 15 20:21:50.085760 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:50.085720    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5x6dz\" (UniqueName: \"kubernetes.io/projected/90ad8ae4-67bf-49ae-bfcb-19d551508307-kube-api-access-5x6dz\") pod \"payload-pre-processing-e2e-inf-b-39c7b0-56db6464b6-6tgv8\" (UID: \"90ad8ae4-67bf-49ae-bfcb-19d551508307\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39c7b0-56db6464b6-6tgv8"
Sep 15 20:21:50.085985 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:50.085774    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90ad8ae4-67bf-49ae-bfcb-19d551508307-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-39c7b0-56db6464b6-6tgv8\" (UID: \"90ad8ae4-67bf-49ae-bfcb-19d551508307\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39c7b0-56db6464b6-6tgv8"
Sep 15 20:21:50.086577 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:50.086547    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90ad8ae4-67bf-49ae-bfcb-19d551508307-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-39c7b0-56db6464b6-6tgv8\" (UID: \"90ad8ae4-67bf-49ae-bfcb-19d551508307\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39c7b0-56db6464b6-6tgv8"
Sep 15 20:21:50.095186 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:50.095152    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5x6dz\" (UniqueName: \"kubernetes.io/projected/90ad8ae4-67bf-49ae-bfcb-19d551508307-kube-api-access-5x6dz\") pod \"payload-pre-processing-e2e-inf-b-39c7b0-56db6464b6-6tgv8\" (UID: \"90ad8ae4-67bf-49ae-bfcb-19d551508307\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39c7b0-56db6464b6-6tgv8"
Sep 15 20:21:50.190528 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:50.190443    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39c7b0-56db6464b6-6tgv8"
Sep 15 20:21:50.355554 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:50.355523    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:50.454686 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:50.454660    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:50.475544 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:50.471765    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-39c7b0-56db6464b6-6tgv8"]
Sep 15 20:21:50.629263 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:50.629199    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:51.296651 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:51.296620    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:51.331873 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:51.331842    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39c7b0-56db6464b6-6tgv8" event={"ID":"90ad8ae4-67bf-49ae-bfcb-19d551508307","Type":"ContainerStarted","Data":"f6c20aaadcdc77bea405895e0e539bd2f6ceaec46ca71625a518d27faee29ff2"}
Sep 15 20:21:51.332069 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:51.331887    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39c7b0-56db6464b6-6tgv8" event={"ID":"90ad8ae4-67bf-49ae-bfcb-19d551508307","Type":"ContainerStarted","Data":"5f0541dcc4fe0b5531084de2f6e67c7b32aa2f24ac1df769525553ffbdee5d10"}
Sep 15 20:21:51.332069 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:51.331939    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39c7b0-56db6464b6-6tgv8"
Sep 15 20:21:51.351312 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:51.351269    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39c7b0-56db6464b6-6tgv8" podStartSLOduration=2.351253092 podStartE2EDuration="2.351253092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:21:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:21:51.349263334 +0000 UTC m=+1025.837727811" watchObservedRunningTime="2026-09-15 20:21:51.351253092 +0000 UTC m=+1025.839717571"
Sep 15 20:21:52.321639 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:52.321593    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:53.158107 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:53.158075    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:54.070881 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:54.070847    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:55.054787 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:55.054754    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:55.770505 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:55.770466    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:56.671076 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:56.671043    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:57.337526 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:57.337497    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:58.132941 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:58.132879    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:58.847324 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:58.847293    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:21:59.566831 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:21:59.566797    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:00.296310 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:00.296274    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:01.225493 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:01.225455    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt"]
Sep 15 20:22:01.230044 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:01.230024    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt"
Sep 15 20:22:01.232553 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:01.232528    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-7e3543\"/\"kube-root-ca.crt\""
Sep 15 20:22:01.233254 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:01.233229    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-7e3543\"/\"simulator-w0-7e3543-kserve-self-signed-certs\""
Sep 15 20:22:01.233361 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:01.233337    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-7e3543\"/\"default-dockercfg-6p8m2\""
Sep 15 20:22:01.233426 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:01.233338    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-7e3543\"/\"openshift-service-ca.crt\""
Sep 15 20:22:01.239097 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:01.239077    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt"]
Sep 15 20:22:01.387633 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:01.387601    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-tmp-dir\") pod \"simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt\" (UID: \"68aa3f0c-93b1-44e7-b21e-3e5353baea2a\") " pod="e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt"
Sep 15 20:22:01.388053 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:01.387656    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-tls-certs\") pod \"simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt\" (UID: \"68aa3f0c-93b1-44e7-b21e-3e5353baea2a\") " pod="e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt"
Sep 15 20:22:01.388053 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:01.387763    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-model-cache\") pod \"simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt\" (UID: \"68aa3f0c-93b1-44e7-b21e-3e5353baea2a\") " pod="e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt"
Sep 15 20:22:01.388053 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:01.387815    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96w7f\" (UniqueName: \"kubernetes.io/projected/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-kube-api-access-96w7f\") pod \"simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt\" (UID: \"68aa3f0c-93b1-44e7-b21e-3e5353baea2a\") " pod="e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt"
Sep 15 20:22:01.388053 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:01.387836    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-dshm\") pod \"simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt\" (UID: \"68aa3f0c-93b1-44e7-b21e-3e5353baea2a\") " pod="e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt"
Sep 15 20:22:01.388053 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:01.387894    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-home\") pod \"simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt\" (UID: \"68aa3f0c-93b1-44e7-b21e-3e5353baea2a\") " pod="e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt"
Sep 15 20:22:01.435119 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:01.435088    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:01.489147 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:01.489038    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-tls-certs\") pod \"simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt\" (UID: \"68aa3f0c-93b1-44e7-b21e-3e5353baea2a\") " pod="e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt"
Sep 15 20:22:01.489147 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:01.489126    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-model-cache\") pod \"simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt\" (UID: \"68aa3f0c-93b1-44e7-b21e-3e5353baea2a\") " pod="e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt"
Sep 15 20:22:01.489390 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:01.489214    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96w7f\" (UniqueName: \"kubernetes.io/projected/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-kube-api-access-96w7f\") pod \"simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt\" (UID: \"68aa3f0c-93b1-44e7-b21e-3e5353baea2a\") " pod="e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt"
Sep 15 20:22:01.489390 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:01.489246    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-dshm\") pod \"simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt\" (UID: \"68aa3f0c-93b1-44e7-b21e-3e5353baea2a\") " pod="e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt"
Sep 15 20:22:01.489390 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:01.489303    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-home\") pod \"simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt\" (UID: \"68aa3f0c-93b1-44e7-b21e-3e5353baea2a\") " pod="e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt"
Sep 15 20:22:01.489390 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:01.489346    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-tmp-dir\") pod \"simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt\" (UID: \"68aa3f0c-93b1-44e7-b21e-3e5353baea2a\") " pod="e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt"
Sep 15 20:22:01.489836 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:01.489808    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-tmp-dir\") pod \"simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt\" (UID: \"68aa3f0c-93b1-44e7-b21e-3e5353baea2a\") " pod="e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt"
Sep 15 20:22:01.490077 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:01.490051    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-home\") pod \"simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt\" (UID: \"68aa3f0c-93b1-44e7-b21e-3e5353baea2a\") " pod="e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt"
Sep 15 20:22:01.490208 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:01.490151    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-model-cache\") pod \"simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt\" (UID: \"68aa3f0c-93b1-44e7-b21e-3e5353baea2a\") " pod="e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt"
Sep 15 20:22:01.492441 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:01.492413    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-dshm\") pod \"simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt\" (UID: \"68aa3f0c-93b1-44e7-b21e-3e5353baea2a\") " pod="e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt"
Sep 15 20:22:01.492679 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:01.492661    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-tls-certs\") pod \"simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt\" (UID: \"68aa3f0c-93b1-44e7-b21e-3e5353baea2a\") " pod="e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt"
Sep 15 20:22:01.497218 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:01.497187    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96w7f\" (UniqueName: \"kubernetes.io/projected/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-kube-api-access-96w7f\") pod \"simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt\" (UID: \"68aa3f0c-93b1-44e7-b21e-3e5353baea2a\") " pod="e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt"
Sep 15 20:22:01.541038 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:01.541002    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt"
Sep 15 20:22:01.817128 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:01.817039    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt"]
Sep 15 20:22:01.819992 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:22:01.819963    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod68aa3f0c_93b1_44e7_b21e_3e5353baea2a.slice/crio-77f527e095777ae1324a8b871bbc9b3ad38a9a93475397b9a101453e7934ef86 WatchSource:0}: Error finding container 77f527e095777ae1324a8b871bbc9b3ad38a9a93475397b9a101453e7934ef86: Status 404 returned error can't find the container with id 77f527e095777ae1324a8b871bbc9b3ad38a9a93475397b9a101453e7934ef86
Sep 15 20:22:02.339138 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:02.339107    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39c7b0-56db6464b6-6tgv8"
Sep 15 20:22:02.387819 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:02.387730    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt" event={"ID":"68aa3f0c-93b1-44e7-b21e-3e5353baea2a","Type":"ContainerStarted","Data":"77668b2a36e91db94d52087a08d1b465ade209ebbad05b97c5105328b727b7fc"}
Sep 15 20:22:02.387819 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:02.387794    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt" event={"ID":"68aa3f0c-93b1-44e7-b21e-3e5353baea2a","Type":"ContainerStarted","Data":"77f527e095777ae1324a8b871bbc9b3ad38a9a93475397b9a101453e7934ef86"}
Sep 15 20:22:02.405903 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:02.405851    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt" podStartSLOduration=1.4058346959999999 podStartE2EDuration="1.405834696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:22:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:22:02.403367783 +0000 UTC m=+1036.891832281" watchObservedRunningTime="2026-09-15 20:22:02.405834696 +0000 UTC m=+1036.894299171"
Sep 15 20:22:04.236570 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:04.236527    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:04.301230 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:04.301188    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:04.402428 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:04.402389    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:04.460798 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:04.460765    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:04.603202 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:04.603115    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:04.864616 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:04.864535    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:04.963277 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:04.963242    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:05.072025 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:05.071997    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:05.154872 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:05.154796    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:05.286440 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:05.286407    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:05.649953 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:05.649901    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:05.810302 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:05.810265    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:06.560765 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:06.560730    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:06.656990 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:06.656955    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:07.376521 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:07.376490    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:08.367453 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:08.367414    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:08.657576 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:08.657498    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:08.756577 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:08.756527    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:09.336264 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:09.336230    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:10.269208 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:10.269180    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:10.373031 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:10.372990    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:10.470453 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:10.470417    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:10.774840 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:10.774813    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:10.848207 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:10.847866    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:10.936859 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:10.936829    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e0b5e00e-d46cdfdb-qn874"]
Sep 15 20:22:10.941630 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:10.941610    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-e0b5e00e-d46cdfdb-qn874"
Sep 15 20:22:10.944260 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:10.944238    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-e0b5e00e-dockercfg-pdw7g\""
Sep 15 20:22:10.944601 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:10.944583    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-e0b5e00e\""
Sep 15 20:22:10.961406 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:10.961381    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e0b5e00e-d46cdfdb-qn874"]
Sep 15 20:22:11.083072 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:11.082987    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9ctx\" (UniqueName: \"kubernetes.io/projected/2fa7ed04-21e0-4353-a88e-69ee47b53098-kube-api-access-m9ctx\") pod \"payload-processing-e2e-migrate-e0b5e00e-d46cdfdb-qn874\" (UID: \"2fa7ed04-21e0-4353-a88e-69ee47b53098\") " pod="openshift-ingress/payload-processing-e2e-migrate-e0b5e00e-d46cdfdb-qn874"
Sep 15 20:22:11.083072 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:11.083060    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fa7ed04-21e0-4353-a88e-69ee47b53098-plugins-config-volume\") pod \"payload-processing-e2e-migrate-e0b5e00e-d46cdfdb-qn874\" (UID: \"2fa7ed04-21e0-4353-a88e-69ee47b53098\") " pod="openshift-ingress/payload-processing-e2e-migrate-e0b5e00e-d46cdfdb-qn874"
Sep 15 20:22:11.184575 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:11.184538    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9ctx\" (UniqueName: \"kubernetes.io/projected/2fa7ed04-21e0-4353-a88e-69ee47b53098-kube-api-access-m9ctx\") pod \"payload-processing-e2e-migrate-e0b5e00e-d46cdfdb-qn874\" (UID: \"2fa7ed04-21e0-4353-a88e-69ee47b53098\") " pod="openshift-ingress/payload-processing-e2e-migrate-e0b5e00e-d46cdfdb-qn874"
Sep 15 20:22:11.184751 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:11.184634    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fa7ed04-21e0-4353-a88e-69ee47b53098-plugins-config-volume\") pod \"payload-processing-e2e-migrate-e0b5e00e-d46cdfdb-qn874\" (UID: \"2fa7ed04-21e0-4353-a88e-69ee47b53098\") " pod="openshift-ingress/payload-processing-e2e-migrate-e0b5e00e-d46cdfdb-qn874"
Sep 15 20:22:11.185332 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:11.185312    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fa7ed04-21e0-4353-a88e-69ee47b53098-plugins-config-volume\") pod \"payload-processing-e2e-migrate-e0b5e00e-d46cdfdb-qn874\" (UID: \"2fa7ed04-21e0-4353-a88e-69ee47b53098\") " pod="openshift-ingress/payload-processing-e2e-migrate-e0b5e00e-d46cdfdb-qn874"
Sep 15 20:22:11.192743 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:11.192726    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9ctx\" (UniqueName: \"kubernetes.io/projected/2fa7ed04-21e0-4353-a88e-69ee47b53098-kube-api-access-m9ctx\") pod \"payload-processing-e2e-migrate-e0b5e00e-d46cdfdb-qn874\" (UID: \"2fa7ed04-21e0-4353-a88e-69ee47b53098\") " pod="openshift-ingress/payload-processing-e2e-migrate-e0b5e00e-d46cdfdb-qn874"
Sep 15 20:22:11.253313 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:11.253282    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-e0b5e00e-d46cdfdb-qn874"
Sep 15 20:22:11.352520 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:11.352417    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:11.528530 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:11.528472    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e0b5e00e-d46cdfdb-qn874"]
Sep 15 20:22:11.530815 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:22:11.530787    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fa7ed04_21e0_4353_a88e_69ee47b53098.slice/crio-1821d9065c9ffaa4720b5584af0264a1a13b451681f4ea74228939a4978f7c15 WatchSource:0}: Error finding container 1821d9065c9ffaa4720b5584af0264a1a13b451681f4ea74228939a4978f7c15: Status 404 returned error can't find the container with id 1821d9065c9ffaa4720b5584af0264a1a13b451681f4ea74228939a4978f7c15
Sep 15 20:22:11.541520 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:11.541487    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt"
Sep 15 20:22:11.541624 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:11.541540    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt"
Sep 15 20:22:11.557952 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:11.557909    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt"
Sep 15 20:22:12.197846 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:12.197809    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:12.445330 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:12.445295    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-e0b5e00e-d46cdfdb-qn874" event={"ID":"2fa7ed04-21e0-4353-a88e-69ee47b53098","Type":"ContainerStarted","Data":"6fbfac647291195103e44e9cd46260f60256a4cf096fd77ea061bff07d60f9de"}
Sep 15 20:22:12.445772 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:12.445386    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-e0b5e00e-d46cdfdb-qn874" event={"ID":"2fa7ed04-21e0-4353-a88e-69ee47b53098","Type":"ContainerStarted","Data":"1821d9065c9ffaa4720b5584af0264a1a13b451681f4ea74228939a4978f7c15"}
Sep 15 20:22:12.445772 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:12.445557    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-e0b5e00e-d46cdfdb-qn874"
Sep 15 20:22:12.456571 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:12.456542    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt"
Sep 15 20:22:12.466719 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:12.466675    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-e0b5e00e-d46cdfdb-qn874" podStartSLOduration=2.466660293 podStartE2EDuration="2.466660293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:22:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:22:12.464806782 +0000 UTC m=+1046.953271258" watchObservedRunningTime="2026-09-15 20:22:12.466660293 +0000 UTC m=+1046.955124769"
Sep 15 20:22:12.941532 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:12.941455    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:13.041712 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:13.041677    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:13.777156 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:13.777122    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:14.476011 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:14.475960    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:15.259648 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:15.259616    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:16.067243 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:16.067207    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:20.804560 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:20.804522    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:23.452493 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:23.452466    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-e0b5e00e-d46cdfdb-qn874"
Sep 15 20:22:32.580662 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:32.580626    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:33.461252 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:33.461224    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:34.252326 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:34.252294    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:34.924529 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:34.924494    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:35.582428 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:35.582395    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:36.289286 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:36.289251    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:36.953418 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:36.953385    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:37.695343 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:37.695310    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:38.422309 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:38.422229    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:39.870141 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:39.870108    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:40.492807 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:40.492773    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:40.511369 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:40.511339    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e0b5e00e-d46cdfdb-qn874"]
Sep 15 20:22:40.511606 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:40.511579    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-e0b5e00e-d46cdfdb-qn874" podUID="2fa7ed04-21e0-4353-a88e-69ee47b53098" containerName="payload-processing" containerID="cri-o://6fbfac647291195103e44e9cd46260f60256a4cf096fd77ea061bff07d60f9de" gracePeriod=30
Sep 15 20:22:40.592619 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:40.592587    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:40.698938 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:40.698895    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:40.756980 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:40.756913    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-e0b5e00e-d46cdfdb-qn874"
Sep 15 20:22:40.821444 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:40.821394    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:40.855469 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:40.855426    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fa7ed04-21e0-4353-a88e-69ee47b53098-plugins-config-volume\") pod \"2fa7ed04-21e0-4353-a88e-69ee47b53098\" (UID: \"2fa7ed04-21e0-4353-a88e-69ee47b53098\") "
Sep 15 20:22:40.855686 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:40.855512    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m9ctx\" (UniqueName: \"kubernetes.io/projected/2fa7ed04-21e0-4353-a88e-69ee47b53098-kube-api-access-m9ctx\") pod \"2fa7ed04-21e0-4353-a88e-69ee47b53098\" (UID: \"2fa7ed04-21e0-4353-a88e-69ee47b53098\") "
Sep 15 20:22:40.855854 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:40.855813    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fa7ed04-21e0-4353-a88e-69ee47b53098-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2fa7ed04-21e0-4353-a88e-69ee47b53098" (UID: "2fa7ed04-21e0-4353-a88e-69ee47b53098"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:22:40.856002 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:40.855984    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fa7ed04-21e0-4353-a88e-69ee47b53098-plugins-config-volume\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:22:40.858474 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:40.858441    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fa7ed04-21e0-4353-a88e-69ee47b53098-kube-api-access-m9ctx" (OuterVolumeSpecName: "kube-api-access-m9ctx") pod "2fa7ed04-21e0-4353-a88e-69ee47b53098" (UID: "2fa7ed04-21e0-4353-a88e-69ee47b53098"). InnerVolumeSpecName "kube-api-access-m9ctx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:22:40.956110 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:40.956080    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:40.956439 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:40.956420    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9ctx\" (UniqueName: \"kubernetes.io/projected/2fa7ed04-21e0-4353-a88e-69ee47b53098-kube-api-access-m9ctx\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:22:41.189480 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:41.189384    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:41.325904 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:41.325872    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:41.614908 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:41.614872    2576 generic.go:356] "Generic (PLEG): container finished" podID="2fa7ed04-21e0-4353-a88e-69ee47b53098" containerID="6fbfac647291195103e44e9cd46260f60256a4cf096fd77ea061bff07d60f9de" exitCode=0
Sep 15 20:22:41.615109 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:41.614913    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-e0b5e00e-d46cdfdb-qn874" event={"ID":"2fa7ed04-21e0-4353-a88e-69ee47b53098","Type":"ContainerDied","Data":"6fbfac647291195103e44e9cd46260f60256a4cf096fd77ea061bff07d60f9de"}
Sep 15 20:22:41.615109 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:41.614959    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-e0b5e00e-d46cdfdb-qn874"
Sep 15 20:22:41.615109 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:41.615007    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-e0b5e00e-d46cdfdb-qn874" event={"ID":"2fa7ed04-21e0-4353-a88e-69ee47b53098","Type":"ContainerDied","Data":"1821d9065c9ffaa4720b5584af0264a1a13b451681f4ea74228939a4978f7c15"}
Sep 15 20:22:41.615109 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:41.615037    2576 scope.go:117] "RemoveContainer" containerID="6fbfac647291195103e44e9cd46260f60256a4cf096fd77ea061bff07d60f9de"
Sep 15 20:22:41.626501 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:41.626467    2576 scope.go:117] "RemoveContainer" containerID="6fbfac647291195103e44e9cd46260f60256a4cf096fd77ea061bff07d60f9de"
Sep 15 20:22:41.626808 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:22:41.626787    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fbfac647291195103e44e9cd46260f60256a4cf096fd77ea061bff07d60f9de\": container with ID starting with 6fbfac647291195103e44e9cd46260f60256a4cf096fd77ea061bff07d60f9de not found: ID does not exist" containerID="6fbfac647291195103e44e9cd46260f60256a4cf096fd77ea061bff07d60f9de"
Sep 15 20:22:41.626878 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:41.626817    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fbfac647291195103e44e9cd46260f60256a4cf096fd77ea061bff07d60f9de"} err="failed to get container status \"6fbfac647291195103e44e9cd46260f60256a4cf096fd77ea061bff07d60f9de\": rpc error: code = NotFound desc = could not find container \"6fbfac647291195103e44e9cd46260f60256a4cf096fd77ea061bff07d60f9de\": container with ID starting with 6fbfac647291195103e44e9cd46260f60256a4cf096fd77ea061bff07d60f9de not found: ID does not exist"
Sep 15 20:22:41.644721 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:41.644694    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e0b5e00e-d46cdfdb-qn874"]
Sep 15 20:22:41.656852 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:41.656831    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e0b5e00e-d46cdfdb-qn874"]
Sep 15 20:22:41.672187 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:41.672161    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:41.836031 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:41.835998    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:42.101363 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:42.101319    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:42.144511 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:42.144471    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fa7ed04-21e0-4353-a88e-69ee47b53098" path="/var/lib/kubelet/pods/2fa7ed04-21e0-4353-a88e-69ee47b53098/volumes"
Sep 15 20:22:42.201415 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:42.201386    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:42.752671 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:42.752638    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:43.465422 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:43.465389    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:43.799310 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:43.799222    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:44.062372 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:44.062286    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:44.389340 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:44.389264    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:45.248709 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:45.248673    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:46.081164 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:46.081140    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:46.184458 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:46.184424    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:46.403156 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:46.403068    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:46.574158 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:46.574131    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:46.794057 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:46.794030    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:46.993643 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:46.993609    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:47.725539 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:47.725493    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:48.580447 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:48.580405    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:49.266799 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:49.266769    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:56.826614 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:56.826583    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6"]
Sep 15 20:22:56.827069 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:56.827055    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fa7ed04-21e0-4353-a88e-69ee47b53098" containerName="payload-processing"
Sep 15 20:22:56.827139 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:56.827071    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fa7ed04-21e0-4353-a88e-69ee47b53098" containerName="payload-processing"
Sep 15 20:22:56.827177 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:56.827150    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fa7ed04-21e0-4353-a88e-69ee47b53098" containerName="payload-processing"
Sep 15 20:22:56.831536 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:56.831518    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6"
Sep 15 20:22:56.834189 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:56.834160    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-30d5aa\"/\"kube-root-ca.crt\""
Sep 15 20:22:56.834189 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:56.834185    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-30d5aa\"/\"autores-model-30d5aa-kserve-self-signed-certs\""
Sep 15 20:22:56.834334 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:56.834160    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-30d5aa\"/\"openshift-service-ca.crt\""
Sep 15 20:22:56.835215 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:56.835198    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-30d5aa\"/\"default-dockercfg-c8xq6\""
Sep 15 20:22:56.839523 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:56.839502    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6"]
Sep 15 20:22:56.846069 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:56.846047    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:22:56.899066 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:56.899035    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-tmp-dir\") pod \"autores-model-30d5aa-kserve-64d4768b87-ldbx6\" (UID: \"25be40f4-bdf3-44ba-b3a8-63e0c489cf50\") " pod="ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6"
Sep 15 20:22:56.899066 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:56.899070    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-home\") pod \"autores-model-30d5aa-kserve-64d4768b87-ldbx6\" (UID: \"25be40f4-bdf3-44ba-b3a8-63e0c489cf50\") " pod="ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6"
Sep 15 20:22:56.899249 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:56.899097    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-dshm\") pod \"autores-model-30d5aa-kserve-64d4768b87-ldbx6\" (UID: \"25be40f4-bdf3-44ba-b3a8-63e0c489cf50\") " pod="ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6"
Sep 15 20:22:56.899249 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:56.899136    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-model-cache\") pod \"autores-model-30d5aa-kserve-64d4768b87-ldbx6\" (UID: \"25be40f4-bdf3-44ba-b3a8-63e0c489cf50\") " pod="ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6"
Sep 15 20:22:56.899249 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:56.899184    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-tls-certs\") pod \"autores-model-30d5aa-kserve-64d4768b87-ldbx6\" (UID: \"25be40f4-bdf3-44ba-b3a8-63e0c489cf50\") " pod="ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6"
Sep 15 20:22:56.899249 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:56.899200    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8mh7\" (UniqueName: \"kubernetes.io/projected/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-kube-api-access-s8mh7\") pod \"autores-model-30d5aa-kserve-64d4768b87-ldbx6\" (UID: \"25be40f4-bdf3-44ba-b3a8-63e0c489cf50\") " pod="ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6"
Sep 15 20:22:56.999971 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:56.999935    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-tls-certs\") pod \"autores-model-30d5aa-kserve-64d4768b87-ldbx6\" (UID: \"25be40f4-bdf3-44ba-b3a8-63e0c489cf50\") " pod="ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6"
Sep 15 20:22:57.000140 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:56.999977    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8mh7\" (UniqueName: \"kubernetes.io/projected/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-kube-api-access-s8mh7\") pod \"autores-model-30d5aa-kserve-64d4768b87-ldbx6\" (UID: \"25be40f4-bdf3-44ba-b3a8-63e0c489cf50\") " pod="ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6"
Sep 15 20:22:57.000140 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:57.000069    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-tmp-dir\") pod \"autores-model-30d5aa-kserve-64d4768b87-ldbx6\" (UID: \"25be40f4-bdf3-44ba-b3a8-63e0c489cf50\") " pod="ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6"
Sep 15 20:22:57.000140 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:57.000102    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-home\") pod \"autores-model-30d5aa-kserve-64d4768b87-ldbx6\" (UID: \"25be40f4-bdf3-44ba-b3a8-63e0c489cf50\") " pod="ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6"
Sep 15 20:22:57.000140 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:57.000128    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-dshm\") pod \"autores-model-30d5aa-kserve-64d4768b87-ldbx6\" (UID: \"25be40f4-bdf3-44ba-b3a8-63e0c489cf50\") " pod="ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6"
Sep 15 20:22:57.000359 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:57.000162    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-model-cache\") pod \"autores-model-30d5aa-kserve-64d4768b87-ldbx6\" (UID: \"25be40f4-bdf3-44ba-b3a8-63e0c489cf50\") " pod="ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6"
Sep 15 20:22:57.000533 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:57.000499    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-tmp-dir\") pod \"autores-model-30d5aa-kserve-64d4768b87-ldbx6\" (UID: \"25be40f4-bdf3-44ba-b3a8-63e0c489cf50\") " pod="ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6"
Sep 15 20:22:57.000644 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:57.000550    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-home\") pod \"autores-model-30d5aa-kserve-64d4768b87-ldbx6\" (UID: \"25be40f4-bdf3-44ba-b3a8-63e0c489cf50\") " pod="ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6"
Sep 15 20:22:57.000644 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:57.000598    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-model-cache\") pod \"autores-model-30d5aa-kserve-64d4768b87-ldbx6\" (UID: \"25be40f4-bdf3-44ba-b3a8-63e0c489cf50\") " pod="ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6"
Sep 15 20:22:57.002312 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:57.002286    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-dshm\") pod \"autores-model-30d5aa-kserve-64d4768b87-ldbx6\" (UID: \"25be40f4-bdf3-44ba-b3a8-63e0c489cf50\") " pod="ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6"
Sep 15 20:22:57.002589 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:57.002573    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-tls-certs\") pod \"autores-model-30d5aa-kserve-64d4768b87-ldbx6\" (UID: \"25be40f4-bdf3-44ba-b3a8-63e0c489cf50\") " pod="ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6"
Sep 15 20:22:57.008932 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:57.008896    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8mh7\" (UniqueName: \"kubernetes.io/projected/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-kube-api-access-s8mh7\") pod \"autores-model-30d5aa-kserve-64d4768b87-ldbx6\" (UID: \"25be40f4-bdf3-44ba-b3a8-63e0c489cf50\") " pod="ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6"
Sep 15 20:22:57.141641 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:57.141577    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6"
Sep 15 20:22:57.418514 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:57.418487    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6"]
Sep 15 20:22:57.421225 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:22:57.421196    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod25be40f4_bdf3_44ba_b3a8_63e0c489cf50.slice/crio-2b038cdaa4347649e6890c735baee09028ec742fcdc40ffc6bb8b6fa5ace39ce WatchSource:0}: Error finding container 2b038cdaa4347649e6890c735baee09028ec742fcdc40ffc6bb8b6fa5ace39ce: Status 404 returned error can't find the container with id 2b038cdaa4347649e6890c735baee09028ec742fcdc40ffc6bb8b6fa5ace39ce
Sep 15 20:22:57.699390 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:57.699311    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6" event={"ID":"25be40f4-bdf3-44ba-b3a8-63e0c489cf50","Type":"ContainerStarted","Data":"456b42fff66a863ea163b7c6b01a9e5a0e21db29da8bf342f91139e9c09080a7"}
Sep 15 20:22:57.699390 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:57.699353    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6" event={"ID":"25be40f4-bdf3-44ba-b3a8-63e0c489cf50","Type":"ContainerStarted","Data":"2b038cdaa4347649e6890c735baee09028ec742fcdc40ffc6bb8b6fa5ace39ce"}
Sep 15 20:22:57.717085 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:22:57.717029    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6" podStartSLOduration=1.717014141 podStartE2EDuration="1.717014141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:22:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:22:57.71576795 +0000 UTC m=+1092.204232428" watchObservedRunningTime="2026-09-15 20:22:57.717014141 +0000 UTC m=+1092.205478617"
Sep 15 20:23:07.142594 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:07.142561    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6"
Sep 15 20:23:07.143159 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:07.142619    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6"
Sep 15 20:23:07.155166 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:07.155142    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6"
Sep 15 20:23:07.779317 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:07.779274    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6"
Sep 15 20:23:08.404859 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:08.404826    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:09.262839 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:09.262805    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:10.299748 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:10.299717    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:11.145523 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:11.145485    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:12.080162 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:12.080128    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:12.781882 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:12.781852    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:13.598767 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:13.598738    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:14.483614 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:14.483580    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:15.283108 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:15.283077    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:15.971656 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:15.971621    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:16.068433 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:16.068402    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:16.160976 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:16.160946    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:16.282505 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:16.282412    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:16.453654 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:16.453626    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:16.840168 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:16.840140    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:17.604905 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:17.604872    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:17.738330 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:17.738292    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:18.052400 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:18.052334    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:18.578704 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:18.578672    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:18.702582 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:18.702542    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:19.364628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:19.364583    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:20.336359 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:20.336310    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:20.980157 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:20.980118    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:21.078611 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:21.078553    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:21.377651 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:21.377566    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:22.060147 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:22.060115    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:22.958773 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:22.958741    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:23.726356 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:23.726322    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:24.406937 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:24.406896    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:25.177501 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:25.177469    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:25.970104 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:25.970066    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:26.052877 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:26.052852    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:26.447304 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:26.447216    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:26.545343 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:26.545296    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:29.274509 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:29.274451    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:30.381275 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:30.381238    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:31.173597 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:31.173558    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:31.836606 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:31.836572    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:32.666624 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:32.666592    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:33.547484 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:33.547432    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:34.699935 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:34.699891    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:35.339176 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:35.339136    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:39.407452 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.407404    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:39.440703 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.440671    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-30d5aa-577f5b65d9-hbrbf"]
Sep 15 20:23:39.441000 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.440968    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-30d5aa-577f5b65d9-hbrbf" podUID="49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672" containerName="payload-processing" containerID="cri-o://8a6cc6fe4423bab6cee2a094ffb7d9ec9b0b69a4be6b726dfc70e2fc57a526dd" gracePeriod=30
Sep 15 20:23:39.449009 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.448983    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-30d5aa-79f75c48cd-fx84v"]
Sep 15 20:23:39.449773 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.449741    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-30d5aa-79f75c48cd-fx84v" podUID="0d95abd6-9b65-4712-8979-449ff240db58" containerName="payload-pre-processing" containerID="cri-o://c68010ec58a3ce5d7cd65d5b96df615e917ada741a5a8de851cc3404ed34894c" gracePeriod=30
Sep 15 20:23:39.506753 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.506727    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:39.607376 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.607338    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:39.687166 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.687098    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:39.728328 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.728307    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-30d5aa-577f5b65d9-hbrbf"
Sep 15 20:23:39.731364 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.731343    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-30d5aa-79f75c48cd-fx84v"
Sep 15 20:23:39.897047 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.897015    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d95abd6-9b65-4712-8979-449ff240db58-plugins-config-volume\") pod \"0d95abd6-9b65-4712-8979-449ff240db58\" (UID: \"0d95abd6-9b65-4712-8979-449ff240db58\") "
Sep 15 20:23:39.897231 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.897075    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7tt8\" (UniqueName: \"kubernetes.io/projected/49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672-kube-api-access-x7tt8\") pod \"49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672\" (UID: \"49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672\") "
Sep 15 20:23:39.897231 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.897102    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpwdb\" (UniqueName: \"kubernetes.io/projected/0d95abd6-9b65-4712-8979-449ff240db58-kube-api-access-cpwdb\") pod \"0d95abd6-9b65-4712-8979-449ff240db58\" (UID: \"0d95abd6-9b65-4712-8979-449ff240db58\") "
Sep 15 20:23:39.897231 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.897162    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672-plugins-config-volume\") pod \"49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672\" (UID: \"49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672\") "
Sep 15 20:23:39.897415 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.897362    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d95abd6-9b65-4712-8979-449ff240db58-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0d95abd6-9b65-4712-8979-449ff240db58" (UID: "0d95abd6-9b65-4712-8979-449ff240db58"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:23:39.897518 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.897499    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d95abd6-9b65-4712-8979-449ff240db58-plugins-config-volume\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:23:39.897564 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.897547    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672" (UID: "49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:23:39.899268 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.899243    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672-kube-api-access-x7tt8" (OuterVolumeSpecName: "kube-api-access-x7tt8") pod "49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672" (UID: "49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672"). InnerVolumeSpecName "kube-api-access-x7tt8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:23:39.899349 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.899266    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d95abd6-9b65-4712-8979-449ff240db58-kube-api-access-cpwdb" (OuterVolumeSpecName: "kube-api-access-cpwdb") pod "0d95abd6-9b65-4712-8979-449ff240db58" (UID: "0d95abd6-9b65-4712-8979-449ff240db58"). InnerVolumeSpecName "kube-api-access-cpwdb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:23:39.921387 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.921355    2576 generic.go:356] "Generic (PLEG): container finished" podID="0d95abd6-9b65-4712-8979-449ff240db58" containerID="c68010ec58a3ce5d7cd65d5b96df615e917ada741a5a8de851cc3404ed34894c" exitCode=0
Sep 15 20:23:39.921483 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.921409    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-30d5aa-79f75c48cd-fx84v"
Sep 15 20:23:39.921483 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.921435    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-30d5aa-79f75c48cd-fx84v" event={"ID":"0d95abd6-9b65-4712-8979-449ff240db58","Type":"ContainerDied","Data":"c68010ec58a3ce5d7cd65d5b96df615e917ada741a5a8de851cc3404ed34894c"}
Sep 15 20:23:39.921581 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.921512    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-30d5aa-79f75c48cd-fx84v" event={"ID":"0d95abd6-9b65-4712-8979-449ff240db58","Type":"ContainerDied","Data":"822c86afc923a14501e56fdd0cd247041977887385dec6fd5191cb866d781cdf"}
Sep 15 20:23:39.921581 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.921536    2576 scope.go:117] "RemoveContainer" containerID="c68010ec58a3ce5d7cd65d5b96df615e917ada741a5a8de851cc3404ed34894c"
Sep 15 20:23:39.922759 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.922737    2576 generic.go:356] "Generic (PLEG): container finished" podID="49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672" containerID="8a6cc6fe4423bab6cee2a094ffb7d9ec9b0b69a4be6b726dfc70e2fc57a526dd" exitCode=0
Sep 15 20:23:39.922828 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.922785    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-30d5aa-577f5b65d9-hbrbf"
Sep 15 20:23:39.922828 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.922807    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-30d5aa-577f5b65d9-hbrbf" event={"ID":"49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672","Type":"ContainerDied","Data":"8a6cc6fe4423bab6cee2a094ffb7d9ec9b0b69a4be6b726dfc70e2fc57a526dd"}
Sep 15 20:23:39.922950 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.922845    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-30d5aa-577f5b65d9-hbrbf" event={"ID":"49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672","Type":"ContainerDied","Data":"319c6643ec3f4c47f159f5835f4bce5f3c45cb1c8d1b2f2bb999603ede4d22a7"}
Sep 15 20:23:39.931519 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.931504    2576 scope.go:117] "RemoveContainer" containerID="c68010ec58a3ce5d7cd65d5b96df615e917ada741a5a8de851cc3404ed34894c"
Sep 15 20:23:39.931799 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:23:39.931779    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c68010ec58a3ce5d7cd65d5b96df615e917ada741a5a8de851cc3404ed34894c\": container with ID starting with c68010ec58a3ce5d7cd65d5b96df615e917ada741a5a8de851cc3404ed34894c not found: ID does not exist" containerID="c68010ec58a3ce5d7cd65d5b96df615e917ada741a5a8de851cc3404ed34894c"
Sep 15 20:23:39.931857 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.931807    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c68010ec58a3ce5d7cd65d5b96df615e917ada741a5a8de851cc3404ed34894c"} err="failed to get container status \"c68010ec58a3ce5d7cd65d5b96df615e917ada741a5a8de851cc3404ed34894c\": rpc error: code = NotFound desc = could not find container \"c68010ec58a3ce5d7cd65d5b96df615e917ada741a5a8de851cc3404ed34894c\": container with ID starting with c68010ec58a3ce5d7cd65d5b96df615e917ada741a5a8de851cc3404ed34894c not found: ID does not exist"
Sep 15 20:23:39.931857 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.931823    2576 scope.go:117] "RemoveContainer" containerID="8a6cc6fe4423bab6cee2a094ffb7d9ec9b0b69a4be6b726dfc70e2fc57a526dd"
Sep 15 20:23:39.940056 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.939911    2576 scope.go:117] "RemoveContainer" containerID="8a6cc6fe4423bab6cee2a094ffb7d9ec9b0b69a4be6b726dfc70e2fc57a526dd"
Sep 15 20:23:39.940346 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:23:39.940319    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a6cc6fe4423bab6cee2a094ffb7d9ec9b0b69a4be6b726dfc70e2fc57a526dd\": container with ID starting with 8a6cc6fe4423bab6cee2a094ffb7d9ec9b0b69a4be6b726dfc70e2fc57a526dd not found: ID does not exist" containerID="8a6cc6fe4423bab6cee2a094ffb7d9ec9b0b69a4be6b726dfc70e2fc57a526dd"
Sep 15 20:23:39.940438 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.940349    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a6cc6fe4423bab6cee2a094ffb7d9ec9b0b69a4be6b726dfc70e2fc57a526dd"} err="failed to get container status \"8a6cc6fe4423bab6cee2a094ffb7d9ec9b0b69a4be6b726dfc70e2fc57a526dd\": rpc error: code = NotFound desc = could not find container \"8a6cc6fe4423bab6cee2a094ffb7d9ec9b0b69a4be6b726dfc70e2fc57a526dd\": container with ID starting with 8a6cc6fe4423bab6cee2a094ffb7d9ec9b0b69a4be6b726dfc70e2fc57a526dd not found: ID does not exist"
Sep 15 20:23:39.947586 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.947565    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-30d5aa-577f5b65d9-hbrbf"]
Sep 15 20:23:39.951524 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.951502    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-30d5aa-577f5b65d9-hbrbf"]
Sep 15 20:23:39.963532 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.963512    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-30d5aa-79f75c48cd-fx84v"]
Sep 15 20:23:39.967120 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.967101    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-30d5aa-79f75c48cd-fx84v"]
Sep 15 20:23:39.998654 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.998633    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpwdb\" (UniqueName: \"kubernetes.io/projected/0d95abd6-9b65-4712-8979-449ff240db58-kube-api-access-cpwdb\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:23:39.998741 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.998656    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672-plugins-config-volume\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:23:39.998741 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:39.998670    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7tt8\" (UniqueName: \"kubernetes.io/projected/49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672-kube-api-access-x7tt8\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:23:40.013324 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:40.013284    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:40.107374 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:40.107332    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:40.145359 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:40.145331    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d95abd6-9b65-4712-8979-449ff240db58" path="/var/lib/kubelet/pods/0d95abd6-9b65-4712-8979-449ff240db58/volumes"
Sep 15 20:23:40.145693 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:40.145681    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672" path="/var/lib/kubelet/pods/49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672/volumes"
Sep 15 20:23:40.774967 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:40.774855    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:41.487825 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:41.487797    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:42.268457 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:42.268421    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:43.086240 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:43.086202    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:43.758965 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:43.758910    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:44.526875 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:44.526841    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:44.994806 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:44.994768    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6"]
Sep 15 20:23:45.091022 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:45.090984    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:45.232790 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:45.232745    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:46.161263 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:46.161235    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:46.965596 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:46.965564    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:48.045463 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:48.045433    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:48.428944 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:48.428846    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:49.258760 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:49.258711    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:49.959989 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:49.959959    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:50.653959 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:50.653909    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:52.686089 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:52.686041    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:52.785465 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:52.785435    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:52.860507 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:52.860480    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:52.993146 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:52.993114    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:53.350091 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:53.350018    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:53.852391 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:53.852337    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:53.946613 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:53.946582    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:54.272850 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:54.272815    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:55.250341 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:55.250310    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:55.355242 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:55.355210    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:55.785153 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:55.785121    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:56.439349 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:56.439320    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:56.822653 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:56.822621    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4"]
Sep 15 20:23:56.823087 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:56.823074    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d95abd6-9b65-4712-8979-449ff240db58" containerName="payload-pre-processing"
Sep 15 20:23:56.823141 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:56.823089    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d95abd6-9b65-4712-8979-449ff240db58" containerName="payload-pre-processing"
Sep 15 20:23:56.823141 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:56.823099    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672" containerName="payload-processing"
Sep 15 20:23:56.823141 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:56.823104    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672" containerName="payload-processing"
Sep 15 20:23:56.823237 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:56.823175    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="49d7ffc8-3eae-4c47-ae6e-bfdbf6d66672" containerName="payload-processing"
Sep 15 20:23:56.823237 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:56.823183    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d95abd6-9b65-4712-8979-449ff240db58" containerName="payload-pre-processing"
Sep 15 20:23:56.827341 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:56.827323    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4"
Sep 15 20:23:56.829995 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:56.829977    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-01ea3b\"/\"test-model-01ea3b-kserve-self-signed-certs\""
Sep 15 20:23:56.830676 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:56.830651    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-01ea3b\"/\"openshift-service-ca.crt\""
Sep 15 20:23:56.830800 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:56.830696    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-01ea3b\"/\"kube-root-ca.crt\""
Sep 15 20:23:56.830800 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:56.830767    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-01ea3b\"/\"default-dockercfg-kxqdk\""
Sep 15 20:23:56.835655 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:56.835630    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4"]
Sep 15 20:23:56.850905 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:56.850881    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z827\" (UniqueName: \"kubernetes.io/projected/a6aa92f4-1431-45b1-85a6-29a91e70acdf-kube-api-access-7z827\") pod \"test-model-01ea3b-kserve-7795b6b86f-wz7x4\" (UID: \"a6aa92f4-1431-45b1-85a6-29a91e70acdf\") " pod="ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4"
Sep 15 20:23:56.851058 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:56.850913    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6aa92f4-1431-45b1-85a6-29a91e70acdf-dshm\") pod \"test-model-01ea3b-kserve-7795b6b86f-wz7x4\" (UID: \"a6aa92f4-1431-45b1-85a6-29a91e70acdf\") " pod="ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4"
Sep 15 20:23:56.851058 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:56.850975    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6aa92f4-1431-45b1-85a6-29a91e70acdf-tls-certs\") pod \"test-model-01ea3b-kserve-7795b6b86f-wz7x4\" (UID: \"a6aa92f4-1431-45b1-85a6-29a91e70acdf\") " pod="ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4"
Sep 15 20:23:56.851058 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:56.850994    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6aa92f4-1431-45b1-85a6-29a91e70acdf-tmp-dir\") pod \"test-model-01ea3b-kserve-7795b6b86f-wz7x4\" (UID: \"a6aa92f4-1431-45b1-85a6-29a91e70acdf\") " pod="ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4"
Sep 15 20:23:56.851058 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:56.851011    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6aa92f4-1431-45b1-85a6-29a91e70acdf-home\") pod \"test-model-01ea3b-kserve-7795b6b86f-wz7x4\" (UID: \"a6aa92f4-1431-45b1-85a6-29a91e70acdf\") " pod="ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4"
Sep 15 20:23:56.851292 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:56.851124    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6aa92f4-1431-45b1-85a6-29a91e70acdf-model-cache\") pod \"test-model-01ea3b-kserve-7795b6b86f-wz7x4\" (UID: \"a6aa92f4-1431-45b1-85a6-29a91e70acdf\") " pod="ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4"
Sep 15 20:23:56.952220 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:56.952191    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6aa92f4-1431-45b1-85a6-29a91e70acdf-model-cache\") pod \"test-model-01ea3b-kserve-7795b6b86f-wz7x4\" (UID: \"a6aa92f4-1431-45b1-85a6-29a91e70acdf\") " pod="ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4"
Sep 15 20:23:56.952365 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:56.952231    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z827\" (UniqueName: \"kubernetes.io/projected/a6aa92f4-1431-45b1-85a6-29a91e70acdf-kube-api-access-7z827\") pod \"test-model-01ea3b-kserve-7795b6b86f-wz7x4\" (UID: \"a6aa92f4-1431-45b1-85a6-29a91e70acdf\") " pod="ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4"
Sep 15 20:23:56.952365 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:56.952257    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6aa92f4-1431-45b1-85a6-29a91e70acdf-dshm\") pod \"test-model-01ea3b-kserve-7795b6b86f-wz7x4\" (UID: \"a6aa92f4-1431-45b1-85a6-29a91e70acdf\") " pod="ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4"
Sep 15 20:23:56.952365 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:56.952287    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6aa92f4-1431-45b1-85a6-29a91e70acdf-tls-certs\") pod \"test-model-01ea3b-kserve-7795b6b86f-wz7x4\" (UID: \"a6aa92f4-1431-45b1-85a6-29a91e70acdf\") " pod="ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4"
Sep 15 20:23:56.952365 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:56.952306    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6aa92f4-1431-45b1-85a6-29a91e70acdf-tmp-dir\") pod \"test-model-01ea3b-kserve-7795b6b86f-wz7x4\" (UID: \"a6aa92f4-1431-45b1-85a6-29a91e70acdf\") " pod="ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4"
Sep 15 20:23:56.952365 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:56.952326    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6aa92f4-1431-45b1-85a6-29a91e70acdf-home\") pod \"test-model-01ea3b-kserve-7795b6b86f-wz7x4\" (UID: \"a6aa92f4-1431-45b1-85a6-29a91e70acdf\") " pod="ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4"
Sep 15 20:23:56.952578 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:56.952556    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6aa92f4-1431-45b1-85a6-29a91e70acdf-model-cache\") pod \"test-model-01ea3b-kserve-7795b6b86f-wz7x4\" (UID: \"a6aa92f4-1431-45b1-85a6-29a91e70acdf\") " pod="ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4"
Sep 15 20:23:56.952692 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:56.952667    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6aa92f4-1431-45b1-85a6-29a91e70acdf-home\") pod \"test-model-01ea3b-kserve-7795b6b86f-wz7x4\" (UID: \"a6aa92f4-1431-45b1-85a6-29a91e70acdf\") " pod="ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4"
Sep 15 20:23:56.952754 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:56.952735    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6aa92f4-1431-45b1-85a6-29a91e70acdf-tmp-dir\") pod \"test-model-01ea3b-kserve-7795b6b86f-wz7x4\" (UID: \"a6aa92f4-1431-45b1-85a6-29a91e70acdf\") " pod="ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4"
Sep 15 20:23:56.954473 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:56.954446    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6aa92f4-1431-45b1-85a6-29a91e70acdf-dshm\") pod \"test-model-01ea3b-kserve-7795b6b86f-wz7x4\" (UID: \"a6aa92f4-1431-45b1-85a6-29a91e70acdf\") " pod="ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4"
Sep 15 20:23:56.954654 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:56.954637    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6aa92f4-1431-45b1-85a6-29a91e70acdf-tls-certs\") pod \"test-model-01ea3b-kserve-7795b6b86f-wz7x4\" (UID: \"a6aa92f4-1431-45b1-85a6-29a91e70acdf\") " pod="ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4"
Sep 15 20:23:56.957492 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:56.957470    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:23:56.959830 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:56.959813    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z827\" (UniqueName: \"kubernetes.io/projected/a6aa92f4-1431-45b1-85a6-29a91e70acdf-kube-api-access-7z827\") pod \"test-model-01ea3b-kserve-7795b6b86f-wz7x4\" (UID: \"a6aa92f4-1431-45b1-85a6-29a91e70acdf\") " pod="ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4"
Sep 15 20:23:57.137724 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:57.137648    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4"
Sep 15 20:23:57.409800 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:57.409720    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4"]
Sep 15 20:23:57.412907 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:23:57.412875    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda6aa92f4_1431_45b1_85a6_29a91e70acdf.slice/crio-09b66946c0b6d0ddcf31fb3d4d70d9874fa92dc5c7dab89b5b311f6a886f263c WatchSource:0}: Error finding container 09b66946c0b6d0ddcf31fb3d4d70d9874fa92dc5c7dab89b5b311f6a886f263c: Status 404 returned error can't find the container with id 09b66946c0b6d0ddcf31fb3d4d70d9874fa92dc5c7dab89b5b311f6a886f263c
Sep 15 20:23:58.017071 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:58.017031    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4" event={"ID":"a6aa92f4-1431-45b1-85a6-29a91e70acdf","Type":"ContainerStarted","Data":"de60b743a2a33bae2b708707697b93d87b7aad7acd978676ad196d7a7f9fdde2"}
Sep 15 20:23:58.017421 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:58.017076    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4" event={"ID":"a6aa92f4-1431-45b1-85a6-29a91e70acdf","Type":"ContainerStarted","Data":"09b66946c0b6d0ddcf31fb3d4d70d9874fa92dc5c7dab89b5b311f6a886f263c"}
Sep 15 20:23:58.036579 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:23:58.036530    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4" podStartSLOduration=2.036515189 podStartE2EDuration="2.036515189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:23:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:23:58.034765491 +0000 UTC m=+1152.523229967" watchObservedRunningTime="2026-09-15 20:23:58.036515189 +0000 UTC m=+1152.524979665"
Sep 15 20:24:00.013982 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:00.013932    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6" podUID="25be40f4-bdf3-44ba-b3a8-63e0c489cf50" containerName="main" containerID="cri-o://456b42fff66a863ea163b7c6b01a9e5a0e21db29da8bf342f91139e9c09080a7" gracePeriod=45
Sep 15 20:24:00.252535 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:00.252510    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6"
Sep 15 20:24:00.281680 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:00.281609    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-tls-certs\") pod \"25be40f4-bdf3-44ba-b3a8-63e0c489cf50\" (UID: \"25be40f4-bdf3-44ba-b3a8-63e0c489cf50\") "
Sep 15 20:24:00.281818 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:00.281704    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8mh7\" (UniqueName: \"kubernetes.io/projected/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-kube-api-access-s8mh7\") pod \"25be40f4-bdf3-44ba-b3a8-63e0c489cf50\" (UID: \"25be40f4-bdf3-44ba-b3a8-63e0c489cf50\") "
Sep 15 20:24:00.281818 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:00.281735    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-home\") pod \"25be40f4-bdf3-44ba-b3a8-63e0c489cf50\" (UID: \"25be40f4-bdf3-44ba-b3a8-63e0c489cf50\") "
Sep 15 20:24:00.281818 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:00.281770    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-model-cache\") pod \"25be40f4-bdf3-44ba-b3a8-63e0c489cf50\" (UID: \"25be40f4-bdf3-44ba-b3a8-63e0c489cf50\") "
Sep 15 20:24:00.282005 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:00.281898    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-dshm\") pod \"25be40f4-bdf3-44ba-b3a8-63e0c489cf50\" (UID: \"25be40f4-bdf3-44ba-b3a8-63e0c489cf50\") "
Sep 15 20:24:00.282063 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:00.282022    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-tmp-dir\") pod \"25be40f4-bdf3-44ba-b3a8-63e0c489cf50\" (UID: \"25be40f4-bdf3-44ba-b3a8-63e0c489cf50\") "
Sep 15 20:24:00.282170 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:00.282144    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-home" (OuterVolumeSpecName: "home") pod "25be40f4-bdf3-44ba-b3a8-63e0c489cf50" (UID: "25be40f4-bdf3-44ba-b3a8-63e0c489cf50"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:24:00.282237 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:00.282165    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-model-cache" (OuterVolumeSpecName: "model-cache") pod "25be40f4-bdf3-44ba-b3a8-63e0c489cf50" (UID: "25be40f4-bdf3-44ba-b3a8-63e0c489cf50"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:24:00.282301 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:00.282278    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "25be40f4-bdf3-44ba-b3a8-63e0c489cf50" (UID: "25be40f4-bdf3-44ba-b3a8-63e0c489cf50"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:24:00.282895 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:00.282873    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-home\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:24:00.283118 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:00.283094    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-model-cache\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:24:00.283118 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:00.283119    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-tmp-dir\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:24:00.283990 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:00.283968    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-kube-api-access-s8mh7" (OuterVolumeSpecName: "kube-api-access-s8mh7") pod "25be40f4-bdf3-44ba-b3a8-63e0c489cf50" (UID: "25be40f4-bdf3-44ba-b3a8-63e0c489cf50"). InnerVolumeSpecName "kube-api-access-s8mh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:24:00.284295 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:00.284273    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-dshm" (OuterVolumeSpecName: "dshm") pod "25be40f4-bdf3-44ba-b3a8-63e0c489cf50" (UID: "25be40f4-bdf3-44ba-b3a8-63e0c489cf50"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:24:00.284454 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:00.284439    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "25be40f4-bdf3-44ba-b3a8-63e0c489cf50" (UID: "25be40f4-bdf3-44ba-b3a8-63e0c489cf50"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:24:00.384000 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:00.383972    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-tls-certs\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:24:00.384000 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:00.383995    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8mh7\" (UniqueName: \"kubernetes.io/projected/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-kube-api-access-s8mh7\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:24:00.384000 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:00.384005    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25be40f4-bdf3-44ba-b3a8-63e0c489cf50-dshm\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:24:01.034296 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:01.034264    2576 generic.go:356] "Generic (PLEG): container finished" podID="25be40f4-bdf3-44ba-b3a8-63e0c489cf50" containerID="456b42fff66a863ea163b7c6b01a9e5a0e21db29da8bf342f91139e9c09080a7" exitCode=0
Sep 15 20:24:01.034703 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:01.034311    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6"
Sep 15 20:24:01.034703 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:01.034357    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6" event={"ID":"25be40f4-bdf3-44ba-b3a8-63e0c489cf50","Type":"ContainerDied","Data":"456b42fff66a863ea163b7c6b01a9e5a0e21db29da8bf342f91139e9c09080a7"}
Sep 15 20:24:01.034703 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:01.034402    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6" event={"ID":"25be40f4-bdf3-44ba-b3a8-63e0c489cf50","Type":"ContainerDied","Data":"2b038cdaa4347649e6890c735baee09028ec742fcdc40ffc6bb8b6fa5ace39ce"}
Sep 15 20:24:01.034703 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:01.034423    2576 scope.go:117] "RemoveContainer" containerID="456b42fff66a863ea163b7c6b01a9e5a0e21db29da8bf342f91139e9c09080a7"
Sep 15 20:24:01.043665 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:01.043648    2576 scope.go:117] "RemoveContainer" containerID="456b42fff66a863ea163b7c6b01a9e5a0e21db29da8bf342f91139e9c09080a7"
Sep 15 20:24:01.043883 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:24:01.043868    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"456b42fff66a863ea163b7c6b01a9e5a0e21db29da8bf342f91139e9c09080a7\": container with ID starting with 456b42fff66a863ea163b7c6b01a9e5a0e21db29da8bf342f91139e9c09080a7 not found: ID does not exist" containerID="456b42fff66a863ea163b7c6b01a9e5a0e21db29da8bf342f91139e9c09080a7"
Sep 15 20:24:01.043957 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:01.043890    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"456b42fff66a863ea163b7c6b01a9e5a0e21db29da8bf342f91139e9c09080a7"} err="failed to get container status \"456b42fff66a863ea163b7c6b01a9e5a0e21db29da8bf342f91139e9c09080a7\": rpc error: code = NotFound desc = could not find container \"456b42fff66a863ea163b7c6b01a9e5a0e21db29da8bf342f91139e9c09080a7\": container with ID starting with 456b42fff66a863ea163b7c6b01a9e5a0e21db29da8bf342f91139e9c09080a7 not found: ID does not exist"
Sep 15 20:24:01.060058 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:01.060035    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6"]
Sep 15 20:24:01.063627 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:01.063606    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-30d5aa/autores-model-30d5aa-kserve-64d4768b87-ldbx6"]
Sep 15 20:24:01.260069 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:01.260035    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:01.363087 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:01.363011    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:01.467960 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:01.467910    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:01.566001 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:01.565967    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:02.033746 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:02.033711    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:02.143643 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:02.143612    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25be40f4-bdf3-44ba-b3a8-63e0c489cf50" path="/var/lib/kubelet/pods/25be40f4-bdf3-44ba-b3a8-63e0c489cf50/volumes"
Sep 15 20:24:06.037214 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:06.037178    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:07.138370 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:07.138342    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4"
Sep 15 20:24:07.138771 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:07.138410    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4"
Sep 15 20:24:07.150947 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:07.150901    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4"
Sep 15 20:24:08.082412 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:08.082385    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4"
Sep 15 20:24:22.039750 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:22.039712    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:22.370582 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:22.370494    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:22.737990 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:22.737958    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:23.063401 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:23.063300    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:23.509329 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:23.509295    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:25.689784 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:25.689753    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:25.978075 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:25.978044    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:26.153874 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:26.153844    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:26.661587 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:26.661554    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:27.504269 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:27.504237    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:27.955403 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:27.955367    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:28.189053 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:28.189012    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:28.891391 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:28.891356    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:28.992245 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:28.992211    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:29.092243 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:29.092213    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:29.349572 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:29.349539    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:29.448770 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:29.448737    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:34.075750 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:34.075718    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:34.179135 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:34.179105    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:34.277011 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:34.276977    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:47.765233 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:47.765199    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:50.135367 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:50.135325    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:50.890451 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:50.890419    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:50.991648 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:50.991613    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:51.650867 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:51.650838    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:52.458223 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:52.458191    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:53.167568 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:53.167536    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:54.065675 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:54.065642    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:54.726811 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:54.726772    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:56.382678 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:56.382652    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:56.487471 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:56.487439    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:24:56.604584 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:24:56.604558    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:25:07.097262 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:07.097227    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:25:07.915310 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:07.915282    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:25:08.249169 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:08.249135    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8"]
Sep 15 20:25:08.249555 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:08.249543    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25be40f4-bdf3-44ba-b3a8-63e0c489cf50" containerName="main"
Sep 15 20:25:08.249600 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:08.249557    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="25be40f4-bdf3-44ba-b3a8-63e0c489cf50" containerName="main"
Sep 15 20:25:08.249648 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:08.249639    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="25be40f4-bdf3-44ba-b3a8-63e0c489cf50" containerName="main"
Sep 15 20:25:08.252404 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:08.252389    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8"
Sep 15 20:25:08.255696 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:08.255674    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-725e67\"/\"default-dockercfg-bff2s\""
Sep 15 20:25:08.256161 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:08.256147    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-725e67\"/\"kube-root-ca.crt\""
Sep 15 20:25:08.256469 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:08.256455    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-725e67\"/\"premium-w4-725e67-kserve-self-signed-certs\""
Sep 15 20:25:08.256519 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:08.256483    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-725e67\"/\"openshift-service-ca.crt\""
Sep 15 20:25:08.264183 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:08.264160    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8"]
Sep 15 20:25:08.276419 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:08.276395    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:25:08.388873 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:08.388842    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5731ba6-6c6a-4538-88b0-3c2e7533b337-home\") pod \"premium-w4-725e67-kserve-64fd8b959b-jkfl8\" (UID: \"e5731ba6-6c6a-4538-88b0-3c2e7533b337\") " pod="e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8"
Sep 15 20:25:08.389086 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:08.388897    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5731ba6-6c6a-4538-88b0-3c2e7533b337-tls-certs\") pod \"premium-w4-725e67-kserve-64fd8b959b-jkfl8\" (UID: \"e5731ba6-6c6a-4538-88b0-3c2e7533b337\") " pod="e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8"
Sep 15 20:25:08.389086 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:08.388951    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5731ba6-6c6a-4538-88b0-3c2e7533b337-model-cache\") pod \"premium-w4-725e67-kserve-64fd8b959b-jkfl8\" (UID: \"e5731ba6-6c6a-4538-88b0-3c2e7533b337\") " pod="e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8"
Sep 15 20:25:08.389086 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:08.388982    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8rln\" (UniqueName: \"kubernetes.io/projected/e5731ba6-6c6a-4538-88b0-3c2e7533b337-kube-api-access-g8rln\") pod \"premium-w4-725e67-kserve-64fd8b959b-jkfl8\" (UID: \"e5731ba6-6c6a-4538-88b0-3c2e7533b337\") " pod="e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8"
Sep 15 20:25:08.389086 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:08.389021    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5731ba6-6c6a-4538-88b0-3c2e7533b337-dshm\") pod \"premium-w4-725e67-kserve-64fd8b959b-jkfl8\" (UID: \"e5731ba6-6c6a-4538-88b0-3c2e7533b337\") " pod="e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8"
Sep 15 20:25:08.389086 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:08.389059    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5731ba6-6c6a-4538-88b0-3c2e7533b337-tmp-dir\") pod \"premium-w4-725e67-kserve-64fd8b959b-jkfl8\" (UID: \"e5731ba6-6c6a-4538-88b0-3c2e7533b337\") " pod="e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8"
Sep 15 20:25:08.489557 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:08.489525    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5731ba6-6c6a-4538-88b0-3c2e7533b337-model-cache\") pod \"premium-w4-725e67-kserve-64fd8b959b-jkfl8\" (UID: \"e5731ba6-6c6a-4538-88b0-3c2e7533b337\") " pod="e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8"
Sep 15 20:25:08.489724 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:08.489563    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8rln\" (UniqueName: \"kubernetes.io/projected/e5731ba6-6c6a-4538-88b0-3c2e7533b337-kube-api-access-g8rln\") pod \"premium-w4-725e67-kserve-64fd8b959b-jkfl8\" (UID: \"e5731ba6-6c6a-4538-88b0-3c2e7533b337\") " pod="e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8"
Sep 15 20:25:08.489724 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:08.489593    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5731ba6-6c6a-4538-88b0-3c2e7533b337-dshm\") pod \"premium-w4-725e67-kserve-64fd8b959b-jkfl8\" (UID: \"e5731ba6-6c6a-4538-88b0-3c2e7533b337\") " pod="e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8"
Sep 15 20:25:08.489724 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:08.489650    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5731ba6-6c6a-4538-88b0-3c2e7533b337-tmp-dir\") pod \"premium-w4-725e67-kserve-64fd8b959b-jkfl8\" (UID: \"e5731ba6-6c6a-4538-88b0-3c2e7533b337\") " pod="e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8"
Sep 15 20:25:08.489724 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:08.489710    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5731ba6-6c6a-4538-88b0-3c2e7533b337-home\") pod \"premium-w4-725e67-kserve-64fd8b959b-jkfl8\" (UID: \"e5731ba6-6c6a-4538-88b0-3c2e7533b337\") " pod="e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8"
Sep 15 20:25:08.489957 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:08.489773    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5731ba6-6c6a-4538-88b0-3c2e7533b337-tls-certs\") pod \"premium-w4-725e67-kserve-64fd8b959b-jkfl8\" (UID: \"e5731ba6-6c6a-4538-88b0-3c2e7533b337\") " pod="e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8"
Sep 15 20:25:08.489957 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:08.489908    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5731ba6-6c6a-4538-88b0-3c2e7533b337-model-cache\") pod \"premium-w4-725e67-kserve-64fd8b959b-jkfl8\" (UID: \"e5731ba6-6c6a-4538-88b0-3c2e7533b337\") " pod="e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8"
Sep 15 20:25:08.490113 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:08.490090    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5731ba6-6c6a-4538-88b0-3c2e7533b337-home\") pod \"premium-w4-725e67-kserve-64fd8b959b-jkfl8\" (UID: \"e5731ba6-6c6a-4538-88b0-3c2e7533b337\") " pod="e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8"
Sep 15 20:25:08.490174 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:08.490139    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5731ba6-6c6a-4538-88b0-3c2e7533b337-tmp-dir\") pod \"premium-w4-725e67-kserve-64fd8b959b-jkfl8\" (UID: \"e5731ba6-6c6a-4538-88b0-3c2e7533b337\") " pod="e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8"
Sep 15 20:25:08.491883 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:08.491861    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5731ba6-6c6a-4538-88b0-3c2e7533b337-dshm\") pod \"premium-w4-725e67-kserve-64fd8b959b-jkfl8\" (UID: \"e5731ba6-6c6a-4538-88b0-3c2e7533b337\") " pod="e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8"
Sep 15 20:25:08.492072 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:08.492056    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5731ba6-6c6a-4538-88b0-3c2e7533b337-tls-certs\") pod \"premium-w4-725e67-kserve-64fd8b959b-jkfl8\" (UID: \"e5731ba6-6c6a-4538-88b0-3c2e7533b337\") " pod="e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8"
Sep 15 20:25:08.497678 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:08.497656    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8rln\" (UniqueName: \"kubernetes.io/projected/e5731ba6-6c6a-4538-88b0-3c2e7533b337-kube-api-access-g8rln\") pod \"premium-w4-725e67-kserve-64fd8b959b-jkfl8\" (UID: \"e5731ba6-6c6a-4538-88b0-3c2e7533b337\") " pod="e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8"
Sep 15 20:25:08.562446 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:08.562389    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8"
Sep 15 20:25:08.833078 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:08.833051    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8"]
Sep 15 20:25:09.390901 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:09.390866    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8" event={"ID":"e5731ba6-6c6a-4538-88b0-3c2e7533b337","Type":"ContainerStarted","Data":"8907815a5b11ae7451f41fdaadfef82d7ec40992049d14aa59f03c8cf5786f36"}
Sep 15 20:25:09.390901 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:09.390907    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8" event={"ID":"e5731ba6-6c6a-4538-88b0-3c2e7533b337","Type":"ContainerStarted","Data":"a175dd6606d425bf61849b55a71b694f52285f50bbd6055fed1d749335f6c0e3"}
Sep 15 20:25:09.408865 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:09.408817    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8" podStartSLOduration=1.4088016780000001 podStartE2EDuration="1.408801678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:25:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:25:09.407819874 +0000 UTC m=+1223.896284350" watchObservedRunningTime="2026-09-15 20:25:09.408801678 +0000 UTC m=+1223.897266191"
Sep 15 20:25:13.946416 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:13.946382    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:25:14.054883 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:14.054845    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:25:14.148962 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:14.148909    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:25:14.208860 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:14.208837    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:25:14.892341 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:14.892302    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:25:14.991356 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:14.991318    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:25:15.091317 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:15.091282    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:25:16.614022 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:16.613990    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:25:18.562901 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:18.562871    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8"
Sep 15 20:25:18.563344 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:18.562977    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8"
Sep 15 20:25:18.575811 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:18.575785    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8"
Sep 15 20:25:19.454708 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:19.454677    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8"
Sep 15 20:25:22.491593 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:22.491532    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:25:22.593147 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:22.593108    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:25:23.166647 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:23.166598    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:25:29.051623 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:29.051586    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:25:29.152135 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:29.152105    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:25:29.282457 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:29.282424    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:25:29.628483 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:29.628451    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:25:29.764917 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:29.764867    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:25:35.390280 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:35.390241    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:25:36.069481 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:36.069452    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:25:36.199977 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:36.199942    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:25:36.960327 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:36.960294    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:25:37.766764 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:37.766730    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:25:38.567267 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:38.567190    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:25:39.245414 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:39.245380    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:25:40.090841 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:40.090809    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:25:40.189295 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:40.189264    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:25:40.288281 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:40.288250    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:25:40.574857 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:40.574826    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:25:40.651741 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:40.651701    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:25:40.687504 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:40.687477    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-dad434fa-547d8b895d-p4hxk"]
Sep 15 20:25:40.691238 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:40.691223    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-dad434fa-547d8b895d-p4hxk"
Sep 15 20:25:40.694088 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:40.694066    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-dad434fa-dockercfg-mfql7\""
Sep 15 20:25:40.694208 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:40.694094    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-dad434fa\""
Sep 15 20:25:40.705117 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:40.705094    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-dad434fa-547d8b895d-p4hxk"]
Sep 15 20:25:40.801811 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:40.801775    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wf6xq\" (UniqueName: \"kubernetes.io/projected/1c243a86-c829-47e9-9f84-54014eb5bcb3-kube-api-access-wf6xq\") pod \"payload-pre-processing-e2e-mt-dad434fa-547d8b895d-p4hxk\" (UID: \"1c243a86-c829-47e9-9f84-54014eb5bcb3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-dad434fa-547d8b895d-p4hxk"
Sep 15 20:25:40.802006 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:40.801914    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c243a86-c829-47e9-9f84-54014eb5bcb3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-dad434fa-547d8b895d-p4hxk\" (UID: \"1c243a86-c829-47e9-9f84-54014eb5bcb3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-dad434fa-547d8b895d-p4hxk"
Sep 15 20:25:40.902472 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:40.902393    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wf6xq\" (UniqueName: \"kubernetes.io/projected/1c243a86-c829-47e9-9f84-54014eb5bcb3-kube-api-access-wf6xq\") pod \"payload-pre-processing-e2e-mt-dad434fa-547d8b895d-p4hxk\" (UID: \"1c243a86-c829-47e9-9f84-54014eb5bcb3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-dad434fa-547d8b895d-p4hxk"
Sep 15 20:25:40.902472 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:40.902464    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c243a86-c829-47e9-9f84-54014eb5bcb3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-dad434fa-547d8b895d-p4hxk\" (UID: \"1c243a86-c829-47e9-9f84-54014eb5bcb3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-dad434fa-547d8b895d-p4hxk"
Sep 15 20:25:40.903089 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:40.903069    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c243a86-c829-47e9-9f84-54014eb5bcb3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-dad434fa-547d8b895d-p4hxk\" (UID: \"1c243a86-c829-47e9-9f84-54014eb5bcb3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-dad434fa-547d8b895d-p4hxk"
Sep 15 20:25:40.910420 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:40.910400    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wf6xq\" (UniqueName: \"kubernetes.io/projected/1c243a86-c829-47e9-9f84-54014eb5bcb3-kube-api-access-wf6xq\") pod \"payload-pre-processing-e2e-mt-dad434fa-547d8b895d-p4hxk\" (UID: \"1c243a86-c829-47e9-9f84-54014eb5bcb3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-dad434fa-547d8b895d-p4hxk"
Sep 15 20:25:41.002536 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:41.002503    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-dad434fa-547d8b895d-p4hxk"
Sep 15 20:25:41.272629 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:41.272603    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-dad434fa-547d8b895d-p4hxk"]
Sep 15 20:25:41.276019 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:25:41.275993    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c243a86_c829_47e9_9f84_54014eb5bcb3.slice/crio-e3eb9ca11ff7c422a4a273c71d642e84d1e5b735b308ef938d6c425a95719dc9 WatchSource:0}: Error finding container e3eb9ca11ff7c422a4a273c71d642e84d1e5b735b308ef938d6c425a95719dc9: Status 404 returned error can't find the container with id e3eb9ca11ff7c422a4a273c71d642e84d1e5b735b308ef938d6c425a95719dc9
Sep 15 20:25:41.564566 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:41.564490    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-dad434fa-547d8b895d-p4hxk" event={"ID":"1c243a86-c829-47e9-9f84-54014eb5bcb3","Type":"ContainerStarted","Data":"d717b73ef1747a0466088c2670be523bf0559716f29cb2df899c87635875d872"}
Sep 15 20:25:41.564566 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:41.564539    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-dad434fa-547d8b895d-p4hxk" event={"ID":"1c243a86-c829-47e9-9f84-54014eb5bcb3","Type":"ContainerStarted","Data":"e3eb9ca11ff7c422a4a273c71d642e84d1e5b735b308ef938d6c425a95719dc9"}
Sep 15 20:25:41.564735 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:41.564580    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-dad434fa-547d8b895d-p4hxk"
Sep 15 20:25:41.581432 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:41.581388    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-dad434fa-547d8b895d-p4hxk" podStartSLOduration=1.581373651 podStartE2EDuration="1.581373651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:25:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:25:41.579061256 +0000 UTC m=+1256.067525731" watchObservedRunningTime="2026-09-15 20:25:41.581373651 +0000 UTC m=+1256.069838126"
Sep 15 20:25:47.727055 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:47.727017    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz"]
Sep 15 20:25:47.730652 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:47.730633    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz"
Sep 15 20:25:47.733827 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:47.733800    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-5c37e0\"/\"explicit-model-5c37e0-kserve-self-signed-certs\""
Sep 15 20:25:47.733999 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:47.733841    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-5c37e0\"/\"default-dockercfg-zkttt\""
Sep 15 20:25:47.733999 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:47.733799    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-5c37e0\"/\"kube-root-ca.crt\""
Sep 15 20:25:47.733999 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:47.733940    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-5c37e0\"/\"openshift-service-ca.crt\""
Sep 15 20:25:47.739644 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:47.739613    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz"]
Sep 15 20:25:47.752967 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:47.752941    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:25:47.866398 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:47.866372    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgmvg\" (UniqueName: \"kubernetes.io/projected/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-kube-api-access-qgmvg\") pod \"explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz\" (UID: \"3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1\") " pod="ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz"
Sep 15 20:25:47.866563 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:47.866426    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-dshm\") pod \"explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz\" (UID: \"3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1\") " pod="ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz"
Sep 15 20:25:47.866563 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:47.866448    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-home\") pod \"explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz\" (UID: \"3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1\") " pod="ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz"
Sep 15 20:25:47.866563 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:47.866528    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-tls-certs\") pod \"explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz\" (UID: \"3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1\") " pod="ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz"
Sep 15 20:25:47.866563 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:47.866560    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-tmp-dir\") pod \"explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz\" (UID: \"3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1\") " pod="ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz"
Sep 15 20:25:47.866780 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:47.866580    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-model-cache\") pod \"explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz\" (UID: \"3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1\") " pod="ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz"
Sep 15 20:25:47.967746 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:47.967708    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-dshm\") pod \"explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz\" (UID: \"3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1\") " pod="ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz"
Sep 15 20:25:47.967885 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:47.967757    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-home\") pod \"explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz\" (UID: \"3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1\") " pod="ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz"
Sep 15 20:25:47.967885 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:47.967790    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-tls-certs\") pod \"explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz\" (UID: \"3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1\") " pod="ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz"
Sep 15 20:25:47.967885 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:47.967818    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-tmp-dir\") pod \"explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz\" (UID: \"3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1\") " pod="ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz"
Sep 15 20:25:47.967885 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:47.967847    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-model-cache\") pod \"explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz\" (UID: \"3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1\") " pod="ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz"
Sep 15 20:25:47.968136 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:47.967989    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgmvg\" (UniqueName: \"kubernetes.io/projected/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-kube-api-access-qgmvg\") pod \"explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz\" (UID: \"3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1\") " pod="ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz"
Sep 15 20:25:47.968243 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:47.968219    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-home\") pod \"explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz\" (UID: \"3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1\") " pod="ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz"
Sep 15 20:25:47.968427 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:47.968403    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-model-cache\") pod \"explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz\" (UID: \"3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1\") " pod="ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz"
Sep 15 20:25:47.968536 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:47.968521    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-tmp-dir\") pod \"explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz\" (UID: \"3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1\") " pod="ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz"
Sep 15 20:25:47.970204 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:47.970183    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-tls-certs\") pod \"explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz\" (UID: \"3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1\") " pod="ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz"
Sep 15 20:25:47.970374 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:47.970358    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-dshm\") pod \"explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz\" (UID: \"3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1\") " pod="ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz"
Sep 15 20:25:47.976159 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:47.976137    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgmvg\" (UniqueName: \"kubernetes.io/projected/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-kube-api-access-qgmvg\") pod \"explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz\" (UID: \"3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1\") " pod="ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz"
Sep 15 20:25:48.041238 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:48.041183    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz"
Sep 15 20:25:48.319642 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:48.319538    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz"]
Sep 15 20:25:48.326414 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:25:48.326384    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3c9e0c2a_f0fa_423c_9e86_38a72c8f2db1.slice/crio-1e156c7dfd66b1bc93053e62b5c567e77b841bc68224db337533802502d3d65d WatchSource:0}: Error finding container 1e156c7dfd66b1bc93053e62b5c567e77b841bc68224db337533802502d3d65d: Status 404 returned error can't find the container with id 1e156c7dfd66b1bc93053e62b5c567e77b841bc68224db337533802502d3d65d
Sep 15 20:25:48.603151 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:48.603073    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz" event={"ID":"3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1","Type":"ContainerStarted","Data":"7cd32ce6ee460324397739d9f84b232c8048b30cf31c83dc65e7f91baada989b"}
Sep 15 20:25:48.603151 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:48.603119    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz" event={"ID":"3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1","Type":"ContainerStarted","Data":"1e156c7dfd66b1bc93053e62b5c567e77b841bc68224db337533802502d3d65d"}
Sep 15 20:25:48.622470 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:48.622421    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz" podStartSLOduration=1.622404401 podStartE2EDuration="1.622404401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:25:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:25:48.620173625 +0000 UTC m=+1263.108638103" watchObservedRunningTime="2026-09-15 20:25:48.622404401 +0000 UTC m=+1263.110868934"
Sep 15 20:25:52.572040 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:52.572007    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-dad434fa-547d8b895d-p4hxk"
Sep 15 20:25:58.041943 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:58.041894    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz"
Sep 15 20:25:58.042325 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:58.042038    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz"
Sep 15 20:25:58.054788 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:58.054767    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz"
Sep 15 20:25:58.666393 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:25:58.666367    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz"
Sep 15 20:26:02.133393 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:02.133365    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:02.789343 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:02.789287    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:03.664859 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:03.664820    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:04.568782 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:04.568753    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:05.256742 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:05.256702    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:06.049742 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:06.049709    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:06.171339 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:06.171311    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:06.734676 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:06.734641    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:06.836342 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:06.836310    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:06.937115 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:06.937082    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:07.259502 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:07.259454    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:10.425254 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:10.425220    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t"]
Sep 15 20:26:10.429073 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:10.429054    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t"
Sep 15 20:26:10.431421 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:10.431397    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-39c7b0\"/\"openshift-service-ca.crt\""
Sep 15 20:26:10.432260 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:10.432243    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-39c7b0\"/\"default-dockercfg-927v6\""
Sep 15 20:26:10.432355 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:10.432260    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-39c7b0\"/\"kube-root-ca.crt\""
Sep 15 20:26:10.432421 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:10.432383    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-39c7b0\"/\"test-model-39c7b0-kserve-self-signed-certs\""
Sep 15 20:26:10.441056 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:10.441035    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t"]
Sep 15 20:26:10.446740 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:10.446717    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:10.471038 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:10.471008    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw48v\" (UniqueName: \"kubernetes.io/projected/60b9e08a-25f4-4c0b-94a1-eabefde3b249-kube-api-access-hw48v\") pod \"test-model-39c7b0-kserve-75cbfddd57-wx97t\" (UID: \"60b9e08a-25f4-4c0b-94a1-eabefde3b249\") " pod="ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t"
Sep 15 20:26:10.471166 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:10.471080    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60b9e08a-25f4-4c0b-94a1-eabefde3b249-home\") pod \"test-model-39c7b0-kserve-75cbfddd57-wx97t\" (UID: \"60b9e08a-25f4-4c0b-94a1-eabefde3b249\") " pod="ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t"
Sep 15 20:26:10.471166 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:10.471127    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60b9e08a-25f4-4c0b-94a1-eabefde3b249-tmp-dir\") pod \"test-model-39c7b0-kserve-75cbfddd57-wx97t\" (UID: \"60b9e08a-25f4-4c0b-94a1-eabefde3b249\") " pod="ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t"
Sep 15 20:26:10.471289 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:10.471197    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60b9e08a-25f4-4c0b-94a1-eabefde3b249-model-cache\") pod \"test-model-39c7b0-kserve-75cbfddd57-wx97t\" (UID: \"60b9e08a-25f4-4c0b-94a1-eabefde3b249\") " pod="ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t"
Sep 15 20:26:10.471289 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:10.471247    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60b9e08a-25f4-4c0b-94a1-eabefde3b249-dshm\") pod \"test-model-39c7b0-kserve-75cbfddd57-wx97t\" (UID: \"60b9e08a-25f4-4c0b-94a1-eabefde3b249\") " pod="ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t"
Sep 15 20:26:10.471394 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:10.471296    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60b9e08a-25f4-4c0b-94a1-eabefde3b249-tls-certs\") pod \"test-model-39c7b0-kserve-75cbfddd57-wx97t\" (UID: \"60b9e08a-25f4-4c0b-94a1-eabefde3b249\") " pod="ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t"
Sep 15 20:26:10.572005 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:10.571974    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60b9e08a-25f4-4c0b-94a1-eabefde3b249-tls-certs\") pod \"test-model-39c7b0-kserve-75cbfddd57-wx97t\" (UID: \"60b9e08a-25f4-4c0b-94a1-eabefde3b249\") " pod="ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t"
Sep 15 20:26:10.572143 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:10.572035    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hw48v\" (UniqueName: \"kubernetes.io/projected/60b9e08a-25f4-4c0b-94a1-eabefde3b249-kube-api-access-hw48v\") pod \"test-model-39c7b0-kserve-75cbfddd57-wx97t\" (UID: \"60b9e08a-25f4-4c0b-94a1-eabefde3b249\") " pod="ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t"
Sep 15 20:26:10.572143 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:10.572101    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60b9e08a-25f4-4c0b-94a1-eabefde3b249-home\") pod \"test-model-39c7b0-kserve-75cbfddd57-wx97t\" (UID: \"60b9e08a-25f4-4c0b-94a1-eabefde3b249\") " pod="ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t"
Sep 15 20:26:10.572224 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:10.572144    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60b9e08a-25f4-4c0b-94a1-eabefde3b249-tmp-dir\") pod \"test-model-39c7b0-kserve-75cbfddd57-wx97t\" (UID: \"60b9e08a-25f4-4c0b-94a1-eabefde3b249\") " pod="ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t"
Sep 15 20:26:10.572224 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:10.572183    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60b9e08a-25f4-4c0b-94a1-eabefde3b249-model-cache\") pod \"test-model-39c7b0-kserve-75cbfddd57-wx97t\" (UID: \"60b9e08a-25f4-4c0b-94a1-eabefde3b249\") " pod="ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t"
Sep 15 20:26:10.572305 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:10.572231    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60b9e08a-25f4-4c0b-94a1-eabefde3b249-dshm\") pod \"test-model-39c7b0-kserve-75cbfddd57-wx97t\" (UID: \"60b9e08a-25f4-4c0b-94a1-eabefde3b249\") " pod="ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t"
Sep 15 20:26:10.572494 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:10.572472    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60b9e08a-25f4-4c0b-94a1-eabefde3b249-tmp-dir\") pod \"test-model-39c7b0-kserve-75cbfddd57-wx97t\" (UID: \"60b9e08a-25f4-4c0b-94a1-eabefde3b249\") " pod="ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t"
Sep 15 20:26:10.572583 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:10.572561    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60b9e08a-25f4-4c0b-94a1-eabefde3b249-home\") pod \"test-model-39c7b0-kserve-75cbfddd57-wx97t\" (UID: \"60b9e08a-25f4-4c0b-94a1-eabefde3b249\") " pod="ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t"
Sep 15 20:26:10.572660 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:10.572642    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60b9e08a-25f4-4c0b-94a1-eabefde3b249-model-cache\") pod \"test-model-39c7b0-kserve-75cbfddd57-wx97t\" (UID: \"60b9e08a-25f4-4c0b-94a1-eabefde3b249\") " pod="ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t"
Sep 15 20:26:10.574394 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:10.574371    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60b9e08a-25f4-4c0b-94a1-eabefde3b249-dshm\") pod \"test-model-39c7b0-kserve-75cbfddd57-wx97t\" (UID: \"60b9e08a-25f4-4c0b-94a1-eabefde3b249\") " pod="ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t"
Sep 15 20:26:10.574677 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:10.574658    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60b9e08a-25f4-4c0b-94a1-eabefde3b249-tls-certs\") pod \"test-model-39c7b0-kserve-75cbfddd57-wx97t\" (UID: \"60b9e08a-25f4-4c0b-94a1-eabefde3b249\") " pod="ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t"
Sep 15 20:26:10.579637 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:10.579617    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw48v\" (UniqueName: \"kubernetes.io/projected/60b9e08a-25f4-4c0b-94a1-eabefde3b249-kube-api-access-hw48v\") pod \"test-model-39c7b0-kserve-75cbfddd57-wx97t\" (UID: \"60b9e08a-25f4-4c0b-94a1-eabefde3b249\") " pod="ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t"
Sep 15 20:26:10.741322 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:10.741298    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t"
Sep 15 20:26:11.013788 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:11.013765    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t"]
Sep 15 20:26:11.015383 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:26:11.015356    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod60b9e08a_25f4_4c0b_94a1_eabefde3b249.slice/crio-1552c7808af0a07c11b4ccf47557eb76e08bff6efbbdb1a8aa45ccefd7dc15ce WatchSource:0}: Error finding container 1552c7808af0a07c11b4ccf47557eb76e08bff6efbbdb1a8aa45ccefd7dc15ce: Status 404 returned error can't find the container with id 1552c7808af0a07c11b4ccf47557eb76e08bff6efbbdb1a8aa45ccefd7dc15ce
Sep 15 20:26:11.724279 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:11.724232    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t" event={"ID":"60b9e08a-25f4-4c0b-94a1-eabefde3b249","Type":"ContainerStarted","Data":"99a074b3e4f297fd0dd7b0b99a4dbae604dc2eefb45a9e0a205192c2cf6e1a5a"}
Sep 15 20:26:11.724279 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:11.724284    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t" event={"ID":"60b9e08a-25f4-4c0b-94a1-eabefde3b249","Type":"ContainerStarted","Data":"1552c7808af0a07c11b4ccf47557eb76e08bff6efbbdb1a8aa45ccefd7dc15ce"}
Sep 15 20:26:11.742282 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:11.742222    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t" podStartSLOduration=1.7422094019999999 podStartE2EDuration="1.742209402s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:26:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:26:11.740577932 +0000 UTC m=+1286.229042409" watchObservedRunningTime="2026-09-15 20:26:11.742209402 +0000 UTC m=+1286.230673918"
Sep 15 20:26:13.224705 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:13.224669    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:13.324652 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:13.324624    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:14.476704 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:14.476674    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:15.256613 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:15.256578    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:16.068466 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:16.068435    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:16.856682 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:16.856653    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:17.465078 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:17.465046    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:17.568004 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:17.567963    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:18.265298 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:18.265264    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:19.066234 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:19.066197    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:20.741724 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:20.741682    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t"
Sep 15 20:26:20.742192 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:20.741758    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t"
Sep 15 20:26:20.754702 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:20.754675    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t"
Sep 15 20:26:20.783977 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:20.783952    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t"
Sep 15 20:26:23.685487 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:23.685453    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:24.427498 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:24.427435    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:24.466975 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:24.462680    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-dad434fa-547d8b895d-p4hxk"]
Sep 15 20:26:24.466975 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:24.463678    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-dad434fa-547d8b895d-p4hxk" podUID="1c243a86-c829-47e9-9f84-54014eb5bcb3" containerName="payload-pre-processing" containerID="cri-o://d717b73ef1747a0466088c2670be523bf0559716f29cb2df899c87635875d872" gracePeriod=30
Sep 15 20:26:24.527076 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:24.527046    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:24.629815 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:24.629784    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:24.729797 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:24.729771    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-dad434fa-547d8b895d-p4hxk"
Sep 15 20:26:24.730688 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:24.730661    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:24.796583 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:24.796545    2576 generic.go:356] "Generic (PLEG): container finished" podID="1c243a86-c829-47e9-9f84-54014eb5bcb3" containerID="d717b73ef1747a0466088c2670be523bf0559716f29cb2df899c87635875d872" exitCode=0
Sep 15 20:26:24.796699 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:24.796591    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-dad434fa-547d8b895d-p4hxk"
Sep 15 20:26:24.796699 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:24.796624    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-dad434fa-547d8b895d-p4hxk" event={"ID":"1c243a86-c829-47e9-9f84-54014eb5bcb3","Type":"ContainerDied","Data":"d717b73ef1747a0466088c2670be523bf0559716f29cb2df899c87635875d872"}
Sep 15 20:26:24.796699 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:24.796678    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-dad434fa-547d8b895d-p4hxk" event={"ID":"1c243a86-c829-47e9-9f84-54014eb5bcb3","Type":"ContainerDied","Data":"e3eb9ca11ff7c422a4a273c71d642e84d1e5b735b308ef938d6c425a95719dc9"}
Sep 15 20:26:24.796817 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:24.796702    2576 scope.go:117] "RemoveContainer" containerID="d717b73ef1747a0466088c2670be523bf0559716f29cb2df899c87635875d872"
Sep 15 20:26:24.805706 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:24.805690    2576 scope.go:117] "RemoveContainer" containerID="d717b73ef1747a0466088c2670be523bf0559716f29cb2df899c87635875d872"
Sep 15 20:26:24.805907 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:24.805890    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wf6xq\" (UniqueName: \"kubernetes.io/projected/1c243a86-c829-47e9-9f84-54014eb5bcb3-kube-api-access-wf6xq\") pod \"1c243a86-c829-47e9-9f84-54014eb5bcb3\" (UID: \"1c243a86-c829-47e9-9f84-54014eb5bcb3\") "
Sep 15 20:26:24.806020 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:26:24.806000    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d717b73ef1747a0466088c2670be523bf0559716f29cb2df899c87635875d872\": container with ID starting with d717b73ef1747a0466088c2670be523bf0559716f29cb2df899c87635875d872 not found: ID does not exist" containerID="d717b73ef1747a0466088c2670be523bf0559716f29cb2df899c87635875d872"
Sep 15 20:26:24.806060 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:24.806030    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d717b73ef1747a0466088c2670be523bf0559716f29cb2df899c87635875d872"} err="failed to get container status \"d717b73ef1747a0466088c2670be523bf0559716f29cb2df899c87635875d872\": rpc error: code = NotFound desc = could not find container \"d717b73ef1747a0466088c2670be523bf0559716f29cb2df899c87635875d872\": container with ID starting with d717b73ef1747a0466088c2670be523bf0559716f29cb2df899c87635875d872 not found: ID does not exist"
Sep 15 20:26:24.806102 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:24.806065    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c243a86-c829-47e9-9f84-54014eb5bcb3-plugins-config-volume\") pod \"1c243a86-c829-47e9-9f84-54014eb5bcb3\" (UID: \"1c243a86-c829-47e9-9f84-54014eb5bcb3\") "
Sep 15 20:26:24.806381 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:24.806357    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c243a86-c829-47e9-9f84-54014eb5bcb3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1c243a86-c829-47e9-9f84-54014eb5bcb3" (UID: "1c243a86-c829-47e9-9f84-54014eb5bcb3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:26:24.807828 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:24.807806    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c243a86-c829-47e9-9f84-54014eb5bcb3-kube-api-access-wf6xq" (OuterVolumeSpecName: "kube-api-access-wf6xq") pod "1c243a86-c829-47e9-9f84-54014eb5bcb3" (UID: "1c243a86-c829-47e9-9f84-54014eb5bcb3"). InnerVolumeSpecName "kube-api-access-wf6xq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:26:24.907610 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:24.907572    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c243a86-c829-47e9-9f84-54014eb5bcb3-plugins-config-volume\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:26:24.907610 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:24.907606    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wf6xq\" (UniqueName: \"kubernetes.io/projected/1c243a86-c829-47e9-9f84-54014eb5bcb3-kube-api-access-wf6xq\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:26:24.951221 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:24.951187    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:25.079443 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:25.079372    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:25.117711 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:25.117684    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-dad434fa-547d8b895d-p4hxk"]
Sep 15 20:26:25.123090 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:25.123069    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-dad434fa-547d8b895d-p4hxk"]
Sep 15 20:26:25.462614 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:25.462588    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:25.562663 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:25.562626    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:25.833588 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:25.833511    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:26.143822 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:26.143746    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c243a86-c829-47e9-9f84-54014eb5bcb3" path="/var/lib/kubelet/pods/1c243a86-c829-47e9-9f84-54014eb5bcb3/volumes"
Sep 15 20:26:26.628895 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:26.628861    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:27.458120 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:27.458089    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:28.182976 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:28.182942    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:29.064390 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:29.064358    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:29.760651 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:29.760590    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:30.825568 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:30.825531    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:30.926127 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:30.926096    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:31.100204 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:31.100120    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:31.411030 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:31.410913    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:31.512331 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:31.512302    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:31.628774 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:31.628744    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:32.714879 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:32.714843    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz"]
Sep 15 20:26:32.736468 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:32.736442    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:33.825693 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:33.825663    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:34.383996 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:34.383965    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:34.460202 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:34.460164    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:34.526271 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:34.526234    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:34.590479 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:34.590449    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:34.681301 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:34.681228    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:34.931944 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:34.931852    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:35.038405 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:35.038375    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:35.768539 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:35.768504    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:36.461214 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:36.461178    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:37.159615 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:37.159578    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:37.775972 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:37.775942    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:37.886656 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:37.886626    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:37.976463 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:37.976432    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:38.119596 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:38.119520    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:40.009649 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:40.009613    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:43.802493 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:43.802456    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:43.901973 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:43.901893    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:44.714981 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:44.714948    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:45.037891 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:45.037815    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:45.370609 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:45.370522    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:47.002985 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:47.002940    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:47.734157 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:47.734123    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz" podUID="3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1" containerName="main" containerID="cri-o://7cd32ce6ee460324397739d9f84b232c8048b30cf31c83dc65e7f91baada989b" gracePeriod=45
Sep 15 20:26:47.840501 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:47.840466    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:47.917567 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:47.917540    2576 generic.go:356] "Generic (PLEG): container finished" podID="3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1" containerID="7cd32ce6ee460324397739d9f84b232c8048b30cf31c83dc65e7f91baada989b" exitCode=0
Sep 15 20:26:47.917704 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:47.917598    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz" event={"ID":"3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1","Type":"ContainerDied","Data":"7cd32ce6ee460324397739d9f84b232c8048b30cf31c83dc65e7f91baada989b"}
Sep 15 20:26:47.935646 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:47.935619    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:48.016893 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:48.016869    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz"
Sep 15 20:26:48.028230 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:48.028206    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-tls-certs\") pod \"3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1\" (UID: \"3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1\") "
Sep 15 20:26:48.028322 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:48.028268    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-tmp-dir\") pod \"3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1\" (UID: \"3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1\") "
Sep 15 20:26:48.028322 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:48.028314    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qgmvg\" (UniqueName: \"kubernetes.io/projected/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-kube-api-access-qgmvg\") pod \"3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1\" (UID: \"3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1\") "
Sep 15 20:26:48.028415 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:48.028340    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-model-cache\") pod \"3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1\" (UID: \"3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1\") "
Sep 15 20:26:48.028415 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:48.028370    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-home\") pod \"3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1\" (UID: \"3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1\") "
Sep 15 20:26:48.028522 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:48.028437    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-dshm\") pod \"3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1\" (UID: \"3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1\") "
Sep 15 20:26:48.028579 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:48.028523    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1" (UID: "3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:26:48.028698 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:48.028672    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-model-cache" (OuterVolumeSpecName: "model-cache") pod "3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1" (UID: "3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:26:48.028766 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:48.028659    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-home" (OuterVolumeSpecName: "home") pod "3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1" (UID: "3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:26:48.028900 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:48.028882    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-tmp-dir\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:26:48.028969 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:48.028902    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-model-cache\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:26:48.028969 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:48.028912    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-home\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:26:48.030987 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:48.030959    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-dshm" (OuterVolumeSpecName: "dshm") pod "3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1" (UID: "3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:26:48.031088 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:48.031017    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1" (UID: "3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:26:48.031088 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:48.031041    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-kube-api-access-qgmvg" (OuterVolumeSpecName: "kube-api-access-qgmvg") pod "3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1" (UID: "3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1"). InnerVolumeSpecName "kube-api-access-qgmvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:26:48.130408 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:48.130378    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgmvg\" (UniqueName: \"kubernetes.io/projected/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-kube-api-access-qgmvg\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:26:48.130408 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:48.130405    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-dshm\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:26:48.130578 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:48.130415    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1-tls-certs\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:26:48.924006 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:48.923972    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz"
Sep 15 20:26:48.924181 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:48.924010    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz" event={"ID":"3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1","Type":"ContainerDied","Data":"1e156c7dfd66b1bc93053e62b5c567e77b841bc68224db337533802502d3d65d"}
Sep 15 20:26:48.924181 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:48.924070    2576 scope.go:117] "RemoveContainer" containerID="7cd32ce6ee460324397739d9f84b232c8048b30cf31c83dc65e7f91baada989b"
Sep 15 20:26:48.949841 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:48.949809    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz"]
Sep 15 20:26:48.950003 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:48.949859    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-5c37e0/explicit-model-5c37e0-kserve-7d84d4cfc9-b2frz"]
Sep 15 20:26:48.959483 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:48.959453    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:49.691200 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:49.691157    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:50.144387 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:50.144354    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1" path="/var/lib/kubelet/pods/3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1/volumes"
Sep 15 20:26:50.445819 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:50.445734    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:51.258225 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:51.258158    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:51.320077 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:51.320039    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:51.497172 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:51.497143    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:51.818127 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:51.818101    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:51.904912 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:51.904868    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:52.247665 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:52.247639    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:53.002989 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:53.002948    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:53.679884 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:53.679855    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:26:54.384291 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:26:54.384237    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:08.425684 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:08.425569    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:12.203787 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:12.203756    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:12.986635 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:12.986601    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:13.654134 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:13.654073    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:14.363811 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:14.363757    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:15.012214 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:15.012158    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:15.772072 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:15.772039    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:16.470597 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:16.470565    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:17.450164 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:17.450130    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:17.625313 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:17.625279    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:17.775936 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:17.775890    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:17.863266 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:17.863235    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:18.132347 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:18.132273    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:18.232678 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:18.232645    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:18.334641 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:18.334608    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:18.455193 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:18.455160    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:18.683715 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:18.683676    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:19.667348 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:19.667308    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:19.767731 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:19.767696    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:19.911577 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:19.911545    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:20.000977 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:20.000952    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:20.104999 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:20.104970    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:21.376075 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:21.376042    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:21.928546 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:21.928514    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:22.020001 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:22.019964    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:22.271880 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:22.271844    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:22.985121 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:22.985089    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:23.752315 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:23.752286    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:24.543762 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:24.543727    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:25.444097 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:25.444059    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:26.429765 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:26.429733    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:26.565256 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:26.565228    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:26.662622 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:26.662585    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:27.029300 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:27.029271    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:27.134194 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:27.134163    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:27.203868 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:27.203836    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f49968-8658b75798-b4dxt"]
Sep 15 20:27:27.204328 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:27.204310    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c243a86-c829-47e9-9f84-54014eb5bcb3" containerName="payload-pre-processing"
Sep 15 20:27:27.204421 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:27.204330    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c243a86-c829-47e9-9f84-54014eb5bcb3" containerName="payload-pre-processing"
Sep 15 20:27:27.204421 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:27.204362    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1" containerName="main"
Sep 15 20:27:27.204421 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:27.204371    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1" containerName="main"
Sep 15 20:27:27.204575 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:27.204496    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c243a86-c829-47e9-9f84-54014eb5bcb3" containerName="payload-pre-processing"
Sep 15 20:27:27.204575 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:27.204512    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c9e0c2a-f0fa-423c-9e86-38a72c8f2db1" containerName="main"
Sep 15 20:27:27.207829 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:27.207808    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f49968-8658b75798-b4dxt"
Sep 15 20:27:27.210005 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:27.209986    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-f49968\""
Sep 15 20:27:27.210127 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:27.210104    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-f49968-dockercfg-zbz9s\""
Sep 15 20:27:27.216940 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:27.216902    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f49968-8658b75798-b4dxt"]
Sep 15 20:27:27.298060 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:27.297976    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76689ad7-ea6b-44cf-bb1b-2190a608e81e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-f49968-8658b75798-b4dxt\" (UID: \"76689ad7-ea6b-44cf-bb1b-2190a608e81e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f49968-8658b75798-b4dxt"
Sep 15 20:27:27.298060 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:27.298013    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwr2z\" (UniqueName: \"kubernetes.io/projected/76689ad7-ea6b-44cf-bb1b-2190a608e81e-kube-api-access-vwr2z\") pod \"payload-pre-processing-e2e-shared-b-w2-f49968-8658b75798-b4dxt\" (UID: \"76689ad7-ea6b-44cf-bb1b-2190a608e81e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f49968-8658b75798-b4dxt"
Sep 15 20:27:27.399236 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:27.399207    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76689ad7-ea6b-44cf-bb1b-2190a608e81e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-f49968-8658b75798-b4dxt\" (UID: \"76689ad7-ea6b-44cf-bb1b-2190a608e81e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f49968-8658b75798-b4dxt"
Sep 15 20:27:27.399379 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:27.399243    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwr2z\" (UniqueName: \"kubernetes.io/projected/76689ad7-ea6b-44cf-bb1b-2190a608e81e-kube-api-access-vwr2z\") pod \"payload-pre-processing-e2e-shared-b-w2-f49968-8658b75798-b4dxt\" (UID: \"76689ad7-ea6b-44cf-bb1b-2190a608e81e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f49968-8658b75798-b4dxt"
Sep 15 20:27:27.399912 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:27.399892    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76689ad7-ea6b-44cf-bb1b-2190a608e81e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-f49968-8658b75798-b4dxt\" (UID: \"76689ad7-ea6b-44cf-bb1b-2190a608e81e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f49968-8658b75798-b4dxt"
Sep 15 20:27:27.407154 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:27.407127    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwr2z\" (UniqueName: \"kubernetes.io/projected/76689ad7-ea6b-44cf-bb1b-2190a608e81e-kube-api-access-vwr2z\") pod \"payload-pre-processing-e2e-shared-b-w2-f49968-8658b75798-b4dxt\" (UID: \"76689ad7-ea6b-44cf-bb1b-2190a608e81e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f49968-8658b75798-b4dxt"
Sep 15 20:27:27.519906 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:27.519877    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f49968-8658b75798-b4dxt"
Sep 15 20:27:27.792572 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:27.792546    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f49968-8658b75798-b4dxt"]
Sep 15 20:27:27.794939 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:27:27.794898    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76689ad7_ea6b_44cf_bb1b_2190a608e81e.slice/crio-2cee0f08bc0e9caf5d057ed6b9e8c272ddb46a6ca1e85b3895f069e989ba3837 WatchSource:0}: Error finding container 2cee0f08bc0e9caf5d057ed6b9e8c272ddb46a6ca1e85b3895f069e989ba3837: Status 404 returned error can't find the container with id 2cee0f08bc0e9caf5d057ed6b9e8c272ddb46a6ca1e85b3895f069e989ba3837
Sep 15 20:27:28.126768 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:28.126671    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f49968-8658b75798-b4dxt" event={"ID":"76689ad7-ea6b-44cf-bb1b-2190a608e81e","Type":"ContainerStarted","Data":"c8e6a2050a38a06a01563fa44eb4cece3eecb303102519ae6982dedf2cf06cb2"}
Sep 15 20:27:28.126768 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:28.126743    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f49968-8658b75798-b4dxt" event={"ID":"76689ad7-ea6b-44cf-bb1b-2190a608e81e","Type":"ContainerStarted","Data":"2cee0f08bc0e9caf5d057ed6b9e8c272ddb46a6ca1e85b3895f069e989ba3837"}
Sep 15 20:27:28.127020 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:28.126865    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f49968-8658b75798-b4dxt"
Sep 15 20:27:28.146988 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:28.146942    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f49968-8658b75798-b4dxt" podStartSLOduration=1.14690606 podStartE2EDuration="1.14690606s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:27:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:27:28.146377629 +0000 UTC m=+1362.634842103" watchObservedRunningTime="2026-09-15 20:27:28.14690606 +0000 UTC m=+1362.635370527"
Sep 15 20:27:36.288786 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:36.288737    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:39.134330 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:39.134300    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f49968-8658b75798-b4dxt"
Sep 15 20:27:45.864259 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:45.864221    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:45.923891 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:45.923858    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:47.422095 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:47.422052    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:48.664067 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:48.664036    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:49.339364 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:49.339326    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:50.078501 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:50.078468    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:50.857374 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:50.857341    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:51.668772 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:51.668743    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:52.012764 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:52.012735    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:52.361294 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:52.361202    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:53.068116 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:53.068083    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:53.426310 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:53.426235    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:53.526213 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:53.526184    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:27:56.142345 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:27:56.142314    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:01.118783 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:01.118748    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:01.422325 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:01.422247    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:01.520061 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:01.520030    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:03.483219 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:03.483187    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:04.253303 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:04.253270    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:04.356273 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:04.356236    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:04.459141 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:04.459105    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:05.199610 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:05.199576    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:06.064282 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:06.064249    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:06.772892 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:06.772864    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:07.448778 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:07.448748    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:08.365515 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:08.365486    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:09.172232 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:09.172200    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:09.235002 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:09.234975    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:09.503894 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:09.503846    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:09.638478 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:09.638448    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:09.733852 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:09.733820    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:09.745506 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:09.745479    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-025acc06-85c48fc6bb-plv6p"]
Sep 15 20:28:09.748537 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:09.748516    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-025acc06-85c48fc6bb-plv6p"
Sep 15 20:28:09.751159 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:09.751139    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-025acc06-dockercfg-6vl92\""
Sep 15 20:28:09.751293 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:09.751165    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-025acc06\""
Sep 15 20:28:09.760950 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:09.760911    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-025acc06-85c48fc6bb-plv6p"]
Sep 15 20:28:09.773220 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:09.773198    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-025acc06-8c68f8b65-frsqn"]
Sep 15 20:28:09.775462 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:09.775447    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-025acc06-8c68f8b65-frsqn"
Sep 15 20:28:09.788143 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:09.788125    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-025acc06-8c68f8b65-frsqn"]
Sep 15 20:28:09.800654 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:09.800632    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7rdw\" (UniqueName: \"kubernetes.io/projected/f2e624f3-e195-41d8-ad2d-36490fdd0edd-kube-api-access-b7rdw\") pod \"payload-processing-e2e-mt-025acc06-8c68f8b65-frsqn\" (UID: \"f2e624f3-e195-41d8-ad2d-36490fdd0edd\") " pod="openshift-ingress/payload-processing-e2e-mt-025acc06-8c68f8b65-frsqn"
Sep 15 20:28:09.800757 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:09.800706    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2e624f3-e195-41d8-ad2d-36490fdd0edd-plugins-config-volume\") pod \"payload-processing-e2e-mt-025acc06-8c68f8b65-frsqn\" (UID: \"f2e624f3-e195-41d8-ad2d-36490fdd0edd\") " pod="openshift-ingress/payload-processing-e2e-mt-025acc06-8c68f8b65-frsqn"
Sep 15 20:28:09.800757 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:09.800752    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05f58653-cd09-44c9-a92a-e282a5c78e83-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-025acc06-85c48fc6bb-plv6p\" (UID: \"05f58653-cd09-44c9-a92a-e282a5c78e83\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-025acc06-85c48fc6bb-plv6p"
Sep 15 20:28:09.800836 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:09.800804    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4769h\" (UniqueName: \"kubernetes.io/projected/05f58653-cd09-44c9-a92a-e282a5c78e83-kube-api-access-4769h\") pod \"payload-pre-processing-e2e-mt-025acc06-85c48fc6bb-plv6p\" (UID: \"05f58653-cd09-44c9-a92a-e282a5c78e83\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-025acc06-85c48fc6bb-plv6p"
Sep 15 20:28:09.901599 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:09.901567    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2e624f3-e195-41d8-ad2d-36490fdd0edd-plugins-config-volume\") pod \"payload-processing-e2e-mt-025acc06-8c68f8b65-frsqn\" (UID: \"f2e624f3-e195-41d8-ad2d-36490fdd0edd\") " pod="openshift-ingress/payload-processing-e2e-mt-025acc06-8c68f8b65-frsqn"
Sep 15 20:28:09.901807 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:09.901611    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05f58653-cd09-44c9-a92a-e282a5c78e83-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-025acc06-85c48fc6bb-plv6p\" (UID: \"05f58653-cd09-44c9-a92a-e282a5c78e83\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-025acc06-85c48fc6bb-plv6p"
Sep 15 20:28:09.901807 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:09.901646    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4769h\" (UniqueName: \"kubernetes.io/projected/05f58653-cd09-44c9-a92a-e282a5c78e83-kube-api-access-4769h\") pod \"payload-pre-processing-e2e-mt-025acc06-85c48fc6bb-plv6p\" (UID: \"05f58653-cd09-44c9-a92a-e282a5c78e83\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-025acc06-85c48fc6bb-plv6p"
Sep 15 20:28:09.901807 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:09.901671    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7rdw\" (UniqueName: \"kubernetes.io/projected/f2e624f3-e195-41d8-ad2d-36490fdd0edd-kube-api-access-b7rdw\") pod \"payload-processing-e2e-mt-025acc06-8c68f8b65-frsqn\" (UID: \"f2e624f3-e195-41d8-ad2d-36490fdd0edd\") " pod="openshift-ingress/payload-processing-e2e-mt-025acc06-8c68f8b65-frsqn"
Sep 15 20:28:09.902292 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:09.902271    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2e624f3-e195-41d8-ad2d-36490fdd0edd-plugins-config-volume\") pod \"payload-processing-e2e-mt-025acc06-8c68f8b65-frsqn\" (UID: \"f2e624f3-e195-41d8-ad2d-36490fdd0edd\") " pod="openshift-ingress/payload-processing-e2e-mt-025acc06-8c68f8b65-frsqn"
Sep 15 20:28:09.902340 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:09.902325    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05f58653-cd09-44c9-a92a-e282a5c78e83-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-025acc06-85c48fc6bb-plv6p\" (UID: \"05f58653-cd09-44c9-a92a-e282a5c78e83\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-025acc06-85c48fc6bb-plv6p"
Sep 15 20:28:09.910161 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:09.910137    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4769h\" (UniqueName: \"kubernetes.io/projected/05f58653-cd09-44c9-a92a-e282a5c78e83-kube-api-access-4769h\") pod \"payload-pre-processing-e2e-mt-025acc06-85c48fc6bb-plv6p\" (UID: \"05f58653-cd09-44c9-a92a-e282a5c78e83\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-025acc06-85c48fc6bb-plv6p"
Sep 15 20:28:09.910254 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:09.910187    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7rdw\" (UniqueName: \"kubernetes.io/projected/f2e624f3-e195-41d8-ad2d-36490fdd0edd-kube-api-access-b7rdw\") pod \"payload-processing-e2e-mt-025acc06-8c68f8b65-frsqn\" (UID: \"f2e624f3-e195-41d8-ad2d-36490fdd0edd\") " pod="openshift-ingress/payload-processing-e2e-mt-025acc06-8c68f8b65-frsqn"
Sep 15 20:28:10.059118 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:10.059037    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-025acc06-85c48fc6bb-plv6p"
Sep 15 20:28:10.085203 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:10.085180    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-025acc06-8c68f8b65-frsqn"
Sep 15 20:28:10.333468 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:10.333442    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-025acc06-85c48fc6bb-plv6p"]
Sep 15 20:28:10.335288 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:28:10.335261    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05f58653_cd09_44c9_a92a_e282a5c78e83.slice/crio-42356f65a844ec09823a9b075a3a0ff2b9cab2284f175356ea0dedf250ef9122 WatchSource:0}: Error finding container 42356f65a844ec09823a9b075a3a0ff2b9cab2284f175356ea0dedf250ef9122: Status 404 returned error can't find the container with id 42356f65a844ec09823a9b075a3a0ff2b9cab2284f175356ea0dedf250ef9122
Sep 15 20:28:10.368671 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:10.368644    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-025acc06-8c68f8b65-frsqn"]
Sep 15 20:28:10.369577 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:28:10.369551    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2e624f3_e195_41d8_ad2d_36490fdd0edd.slice/crio-fc23699c853a5a37214ed8ca234c6df0a1cc5e0b36c16520bb49c9515f9d19e7 WatchSource:0}: Error finding container fc23699c853a5a37214ed8ca234c6df0a1cc5e0b36c16520bb49c9515f9d19e7: Status 404 returned error can't find the container with id fc23699c853a5a37214ed8ca234c6df0a1cc5e0b36c16520bb49c9515f9d19e7
Sep 15 20:28:11.348237 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:11.348206    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-025acc06-85c48fc6bb-plv6p" event={"ID":"05f58653-cd09-44c9-a92a-e282a5c78e83","Type":"ContainerStarted","Data":"0e85809f6c6dfd6b94ef7dd40cc3f999052a90f4778f73398f9097c2ccf6647d"}
Sep 15 20:28:11.348659 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:11.348297    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-025acc06-85c48fc6bb-plv6p"
Sep 15 20:28:11.348659 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:11.348351    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-025acc06-85c48fc6bb-plv6p" event={"ID":"05f58653-cd09-44c9-a92a-e282a5c78e83","Type":"ContainerStarted","Data":"42356f65a844ec09823a9b075a3a0ff2b9cab2284f175356ea0dedf250ef9122"}
Sep 15 20:28:11.349946 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:11.349908    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-025acc06-8c68f8b65-frsqn" event={"ID":"f2e624f3-e195-41d8-ad2d-36490fdd0edd","Type":"ContainerStarted","Data":"aa87c4817ebd8b9e5fdd2a18d79b2ace721101caa71a71384f7fad83ca7e6c98"}
Sep 15 20:28:11.350070 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:11.349969    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-025acc06-8c68f8b65-frsqn"
Sep 15 20:28:11.350070 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:11.349993    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-025acc06-8c68f8b65-frsqn" event={"ID":"f2e624f3-e195-41d8-ad2d-36490fdd0edd","Type":"ContainerStarted","Data":"fc23699c853a5a37214ed8ca234c6df0a1cc5e0b36c16520bb49c9515f9d19e7"}
Sep 15 20:28:11.366399 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:11.366350    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-025acc06-85c48fc6bb-plv6p" podStartSLOduration=2.366331053 podStartE2EDuration="2.366331053s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:28:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:28:11.365030773 +0000 UTC m=+1405.853495250" watchObservedRunningTime="2026-09-15 20:28:11.366331053 +0000 UTC m=+1405.854795535"
Sep 15 20:28:11.380320 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:11.380249    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-025acc06-8c68f8b65-frsqn" podStartSLOduration=2.3802331629999998 podStartE2EDuration="2.380233163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:28:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:28:11.379904457 +0000 UTC m=+1405.868368933" watchObservedRunningTime="2026-09-15 20:28:11.380233163 +0000 UTC m=+1405.868697640"
Sep 15 20:28:22.357753 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:22.357720    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-025acc06-85c48fc6bb-plv6p"
Sep 15 20:28:22.358217 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:22.357791    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-025acc06-8c68f8b65-frsqn"
Sep 15 20:28:29.218073 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:29.218039    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz"]
Sep 15 20:28:29.221823 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:29.221806    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz"
Sep 15 20:28:29.224978 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:29.224950    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-6733e9\"/\"kube-root-ca.crt\""
Sep 15 20:28:29.225117 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:29.225045    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-6733e9\"/\"default-dockercfg-j9pbx\""
Sep 15 20:28:29.225182 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:29.225162    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-6733e9\"/\"noten-model-6733e9-kserve-self-signed-certs\""
Sep 15 20:28:29.225233 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:29.225171    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-6733e9\"/\"openshift-service-ca.crt\""
Sep 15 20:28:29.228121 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:29.228102    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz"]
Sep 15 20:28:29.381806 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:29.381775    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/354151e6-3aa2-484f-99be-7cb3455045c7-dshm\") pod \"noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz\" (UID: \"354151e6-3aa2-484f-99be-7cb3455045c7\") " pod="ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz"
Sep 15 20:28:29.381978 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:29.381810    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/354151e6-3aa2-484f-99be-7cb3455045c7-home\") pod \"noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz\" (UID: \"354151e6-3aa2-484f-99be-7cb3455045c7\") " pod="ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz"
Sep 15 20:28:29.381978 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:29.381939    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/354151e6-3aa2-484f-99be-7cb3455045c7-tls-certs\") pod \"noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz\" (UID: \"354151e6-3aa2-484f-99be-7cb3455045c7\") " pod="ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz"
Sep 15 20:28:29.382077 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:29.381982    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/354151e6-3aa2-484f-99be-7cb3455045c7-tmp-dir\") pod \"noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz\" (UID: \"354151e6-3aa2-484f-99be-7cb3455045c7\") " pod="ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz"
Sep 15 20:28:29.382077 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:29.382057    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnql7\" (UniqueName: \"kubernetes.io/projected/354151e6-3aa2-484f-99be-7cb3455045c7-kube-api-access-qnql7\") pod \"noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz\" (UID: \"354151e6-3aa2-484f-99be-7cb3455045c7\") " pod="ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz"
Sep 15 20:28:29.382151 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:29.382093    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/354151e6-3aa2-484f-99be-7cb3455045c7-model-cache\") pod \"noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz\" (UID: \"354151e6-3aa2-484f-99be-7cb3455045c7\") " pod="ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz"
Sep 15 20:28:29.438849 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:29.438818    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:29.483454 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:29.483377    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/354151e6-3aa2-484f-99be-7cb3455045c7-tls-certs\") pod \"noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz\" (UID: \"354151e6-3aa2-484f-99be-7cb3455045c7\") " pod="ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz"
Sep 15 20:28:29.483454 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:29.483427    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/354151e6-3aa2-484f-99be-7cb3455045c7-tmp-dir\") pod \"noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz\" (UID: \"354151e6-3aa2-484f-99be-7cb3455045c7\") " pod="ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz"
Sep 15 20:28:29.483659 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:29.483496    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnql7\" (UniqueName: \"kubernetes.io/projected/354151e6-3aa2-484f-99be-7cb3455045c7-kube-api-access-qnql7\") pod \"noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz\" (UID: \"354151e6-3aa2-484f-99be-7cb3455045c7\") " pod="ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz"
Sep 15 20:28:29.483659 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:29.483618    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/354151e6-3aa2-484f-99be-7cb3455045c7-model-cache\") pod \"noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz\" (UID: \"354151e6-3aa2-484f-99be-7cb3455045c7\") " pod="ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz"
Sep 15 20:28:29.483772 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:29.483670    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/354151e6-3aa2-484f-99be-7cb3455045c7-dshm\") pod \"noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz\" (UID: \"354151e6-3aa2-484f-99be-7cb3455045c7\") " pod="ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz"
Sep 15 20:28:29.483772 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:29.483703    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/354151e6-3aa2-484f-99be-7cb3455045c7-home\") pod \"noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz\" (UID: \"354151e6-3aa2-484f-99be-7cb3455045c7\") " pod="ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz"
Sep 15 20:28:29.483975 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:29.483950    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/354151e6-3aa2-484f-99be-7cb3455045c7-tmp-dir\") pod \"noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz\" (UID: \"354151e6-3aa2-484f-99be-7cb3455045c7\") " pod="ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz"
Sep 15 20:28:29.484056 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:29.483994    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/354151e6-3aa2-484f-99be-7cb3455045c7-model-cache\") pod \"noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz\" (UID: \"354151e6-3aa2-484f-99be-7cb3455045c7\") " pod="ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz"
Sep 15 20:28:29.484095 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:29.484063    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/354151e6-3aa2-484f-99be-7cb3455045c7-home\") pod \"noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz\" (UID: \"354151e6-3aa2-484f-99be-7cb3455045c7\") " pod="ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz"
Sep 15 20:28:29.485703 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:29.485682    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/354151e6-3aa2-484f-99be-7cb3455045c7-dshm\") pod \"noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz\" (UID: \"354151e6-3aa2-484f-99be-7cb3455045c7\") " pod="ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz"
Sep 15 20:28:29.485995 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:29.485981    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/354151e6-3aa2-484f-99be-7cb3455045c7-tls-certs\") pod \"noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz\" (UID: \"354151e6-3aa2-484f-99be-7cb3455045c7\") " pod="ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz"
Sep 15 20:28:29.491699 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:29.491677    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnql7\" (UniqueName: \"kubernetes.io/projected/354151e6-3aa2-484f-99be-7cb3455045c7-kube-api-access-qnql7\") pod \"noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz\" (UID: \"354151e6-3aa2-484f-99be-7cb3455045c7\") " pod="ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz"
Sep 15 20:28:29.531827 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:29.531803    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz"
Sep 15 20:28:29.801279 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:29.801252    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz"]
Sep 15 20:28:29.803834 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:28:29.803802    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod354151e6_3aa2_484f_99be_7cb3455045c7.slice/crio-e1092e4da24acbf5ebb33568f064460ef22ee7b7954735b2a9b57249bb92ae35 WatchSource:0}: Error finding container e1092e4da24acbf5ebb33568f064460ef22ee7b7954735b2a9b57249bb92ae35: Status 404 returned error can't find the container with id e1092e4da24acbf5ebb33568f064460ef22ee7b7954735b2a9b57249bb92ae35
Sep 15 20:28:30.455565 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:30.455520    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz" event={"ID":"354151e6-3aa2-484f-99be-7cb3455045c7","Type":"ContainerStarted","Data":"7b124e3801fb63f63657d66f2af9c96e9b04e25a16a4c9af5f998313f9f3fcb4"}
Sep 15 20:28:30.455565 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:30.455568    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz" event={"ID":"354151e6-3aa2-484f-99be-7cb3455045c7","Type":"ContainerStarted","Data":"e1092e4da24acbf5ebb33568f064460ef22ee7b7954735b2a9b57249bb92ae35"}
Sep 15 20:28:30.474708 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:30.474659    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz" podStartSLOduration=1.474644505 podStartE2EDuration="1.474644505s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:28:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:28:30.474069448 +0000 UTC m=+1424.962533916" watchObservedRunningTime="2026-09-15 20:28:30.474644505 +0000 UTC m=+1424.963108981"
Sep 15 20:28:31.167234 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:31.167190    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:31.856326 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:31.856269    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:32.559310 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:32.559279    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:32.613719 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:32.613682    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4"]
Sep 15 20:28:32.636618 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:32.636595    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:33.363454 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:33.363422    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:34.051335 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:34.051262    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:34.273614 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:34.273570    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:35.797993 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:35.797948    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:35.899064 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:35.899032    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:36.679850 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:36.679815    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:39.532749 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:39.532668    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz"
Sep 15 20:28:39.532749 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:39.532710    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz"
Sep 15 20:28:39.545110 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:39.545087    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz"
Sep 15 20:28:40.160196 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:40.160162    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t"]
Sep 15 20:28:40.177642 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:40.177595    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:40.467200 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:40.467163    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:40.525714 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:40.525678    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz"
Sep 15 20:28:41.357374 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:41.357336    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:42.020458 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:42.020422    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:42.868803 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:42.868772    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:43.561631 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:43.561587    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:44.259625 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:44.259592    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:45.073185 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:45.073138    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:45.728153 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:45.728111    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:46.384115 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:46.384083    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:47.632931 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:47.632889    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4" podUID="a6aa92f4-1431-45b1-85a6-29a91e70acdf" containerName="main" containerID="cri-o://de60b743a2a33bae2b708707697b93d87b7aad7acd978676ad196d7a7f9fdde2" gracePeriod=45
Sep 15 20:28:47.874901 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:47.874876    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4"
Sep 15 20:28:47.966003 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:47.965978    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6aa92f4-1431-45b1-85a6-29a91e70acdf-tmp-dir\") pod \"a6aa92f4-1431-45b1-85a6-29a91e70acdf\" (UID: \"a6aa92f4-1431-45b1-85a6-29a91e70acdf\") "
Sep 15 20:28:47.966126 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:47.966033    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6aa92f4-1431-45b1-85a6-29a91e70acdf-home\") pod \"a6aa92f4-1431-45b1-85a6-29a91e70acdf\" (UID: \"a6aa92f4-1431-45b1-85a6-29a91e70acdf\") "
Sep 15 20:28:47.966126 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:47.966064    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6aa92f4-1431-45b1-85a6-29a91e70acdf-model-cache\") pod \"a6aa92f4-1431-45b1-85a6-29a91e70acdf\" (UID: \"a6aa92f4-1431-45b1-85a6-29a91e70acdf\") "
Sep 15 20:28:47.966126 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:47.966109    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6aa92f4-1431-45b1-85a6-29a91e70acdf-dshm\") pod \"a6aa92f4-1431-45b1-85a6-29a91e70acdf\" (UID: \"a6aa92f4-1431-45b1-85a6-29a91e70acdf\") "
Sep 15 20:28:47.966237 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:47.966173    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6aa92f4-1431-45b1-85a6-29a91e70acdf-tls-certs\") pod \"a6aa92f4-1431-45b1-85a6-29a91e70acdf\" (UID: \"a6aa92f4-1431-45b1-85a6-29a91e70acdf\") "
Sep 15 20:28:47.966237 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:47.966220    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6aa92f4-1431-45b1-85a6-29a91e70acdf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a6aa92f4-1431-45b1-85a6-29a91e70acdf" (UID: "a6aa92f4-1431-45b1-85a6-29a91e70acdf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:28:47.966343 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:47.966225    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7z827\" (UniqueName: \"kubernetes.io/projected/a6aa92f4-1431-45b1-85a6-29a91e70acdf-kube-api-access-7z827\") pod \"a6aa92f4-1431-45b1-85a6-29a91e70acdf\" (UID: \"a6aa92f4-1431-45b1-85a6-29a91e70acdf\") "
Sep 15 20:28:47.966343 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:47.966335    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6aa92f4-1431-45b1-85a6-29a91e70acdf-model-cache" (OuterVolumeSpecName: "model-cache") pod "a6aa92f4-1431-45b1-85a6-29a91e70acdf" (UID: "a6aa92f4-1431-45b1-85a6-29a91e70acdf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:28:47.966457 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:47.966348    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6aa92f4-1431-45b1-85a6-29a91e70acdf-home" (OuterVolumeSpecName: "home") pod "a6aa92f4-1431-45b1-85a6-29a91e70acdf" (UID: "a6aa92f4-1431-45b1-85a6-29a91e70acdf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:28:47.966571 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:47.966549    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6aa92f4-1431-45b1-85a6-29a91e70acdf-tmp-dir\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:28:47.966571 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:47.966569    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6aa92f4-1431-45b1-85a6-29a91e70acdf-home\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:28:47.966571 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:47.966578    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6aa92f4-1431-45b1-85a6-29a91e70acdf-model-cache\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:28:47.968307 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:47.968284    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6aa92f4-1431-45b1-85a6-29a91e70acdf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a6aa92f4-1431-45b1-85a6-29a91e70acdf" (UID: "a6aa92f4-1431-45b1-85a6-29a91e70acdf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:28:47.968390 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:47.968346    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6aa92f4-1431-45b1-85a6-29a91e70acdf-dshm" (OuterVolumeSpecName: "dshm") pod "a6aa92f4-1431-45b1-85a6-29a91e70acdf" (UID: "a6aa92f4-1431-45b1-85a6-29a91e70acdf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:28:47.968390 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:47.968354    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6aa92f4-1431-45b1-85a6-29a91e70acdf-kube-api-access-7z827" (OuterVolumeSpecName: "kube-api-access-7z827") pod "a6aa92f4-1431-45b1-85a6-29a91e70acdf" (UID: "a6aa92f4-1431-45b1-85a6-29a91e70acdf"). InnerVolumeSpecName "kube-api-access-7z827". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:28:48.067024 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:48.067000    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6aa92f4-1431-45b1-85a6-29a91e70acdf-dshm\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:28:48.067024 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:48.067021    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6aa92f4-1431-45b1-85a6-29a91e70acdf-tls-certs\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:28:48.067170 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:48.067032    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7z827\" (UniqueName: \"kubernetes.io/projected/a6aa92f4-1431-45b1-85a6-29a91e70acdf-kube-api-access-7z827\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:28:48.558432 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:48.558398    2576 generic.go:356] "Generic (PLEG): container finished" podID="a6aa92f4-1431-45b1-85a6-29a91e70acdf" containerID="de60b743a2a33bae2b708707697b93d87b7aad7acd978676ad196d7a7f9fdde2" exitCode=0
Sep 15 20:28:48.558596 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:48.558455    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4"
Sep 15 20:28:48.558596 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:48.558480    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4" event={"ID":"a6aa92f4-1431-45b1-85a6-29a91e70acdf","Type":"ContainerDied","Data":"de60b743a2a33bae2b708707697b93d87b7aad7acd978676ad196d7a7f9fdde2"}
Sep 15 20:28:48.558596 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:48.558527    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4" event={"ID":"a6aa92f4-1431-45b1-85a6-29a91e70acdf","Type":"ContainerDied","Data":"09b66946c0b6d0ddcf31fb3d4d70d9874fa92dc5c7dab89b5b311f6a886f263c"}
Sep 15 20:28:48.558596 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:48.558552    2576 scope.go:117] "RemoveContainer" containerID="de60b743a2a33bae2b708707697b93d87b7aad7acd978676ad196d7a7f9fdde2"
Sep 15 20:28:48.568880 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:48.568859    2576 scope.go:117] "RemoveContainer" containerID="de60b743a2a33bae2b708707697b93d87b7aad7acd978676ad196d7a7f9fdde2"
Sep 15 20:28:48.569215 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:28:48.569196    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de60b743a2a33bae2b708707697b93d87b7aad7acd978676ad196d7a7f9fdde2\": container with ID starting with de60b743a2a33bae2b708707697b93d87b7aad7acd978676ad196d7a7f9fdde2 not found: ID does not exist" containerID="de60b743a2a33bae2b708707697b93d87b7aad7acd978676ad196d7a7f9fdde2"
Sep 15 20:28:48.569278 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:48.569234    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de60b743a2a33bae2b708707697b93d87b7aad7acd978676ad196d7a7f9fdde2"} err="failed to get container status \"de60b743a2a33bae2b708707697b93d87b7aad7acd978676ad196d7a7f9fdde2\": rpc error: code = NotFound desc = could not find container \"de60b743a2a33bae2b708707697b93d87b7aad7acd978676ad196d7a7f9fdde2\": container with ID starting with de60b743a2a33bae2b708707697b93d87b7aad7acd978676ad196d7a7f9fdde2 not found: ID does not exist"
Sep 15 20:28:48.576213 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:48.576190    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4"]
Sep 15 20:28:48.582618 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:48.582599    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-01ea3b/test-model-01ea3b-kserve-7795b6b86f-wz7x4"]
Sep 15 20:28:50.144874 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:50.144832    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6aa92f4-1431-45b1-85a6-29a91e70acdf" path="/var/lib/kubelet/pods/a6aa92f4-1431-45b1-85a6-29a91e70acdf/volumes"
Sep 15 20:28:50.885975 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:50.885914    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:50.917605 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:50.917567    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-01ea3b-5cfd8d6bdf-ln7n4"]
Sep 15 20:28:50.917841 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:50.917810    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-01ea3b-5cfd8d6bdf-ln7n4" podUID="4fad2e57-3655-4526-a43e-8d57820aba63" containerName="payload-processing" containerID="cri-o://a1a06a9254dd55886965af730907ce8cf7dcb26201f500bde9676797d3af2a70" gracePeriod=30
Sep 15 20:28:50.925126 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:50.925096    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-01ea3b-7c64f65746-h5cxj"]
Sep 15 20:28:50.925359 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:50.925315    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-01ea3b-7c64f65746-h5cxj" podUID="5a5673a7-326b-441f-9fd6-2a3560e16199" containerName="payload-pre-processing" containerID="cri-o://99d76e9629e25b0c841cf64dca7f97fd2b30045fa56d40f50aaa53cf14872600" gracePeriod=30
Sep 15 20:28:50.958863 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:50.958824    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:51.020473 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.020448    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:51.083618 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.083559    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:51.174767 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.174741    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-01ea3b-5cfd8d6bdf-ln7n4"
Sep 15 20:28:51.208083 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.208053    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-01ea3b-7c64f65746-h5cxj"
Sep 15 20:28:51.293832 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.293806    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fad2e57-3655-4526-a43e-8d57820aba63-plugins-config-volume\") pod \"4fad2e57-3655-4526-a43e-8d57820aba63\" (UID: \"4fad2e57-3655-4526-a43e-8d57820aba63\") "
Sep 15 20:28:51.293976 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.293880    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a5673a7-326b-441f-9fd6-2a3560e16199-plugins-config-volume\") pod \"5a5673a7-326b-441f-9fd6-2a3560e16199\" (UID: \"5a5673a7-326b-441f-9fd6-2a3560e16199\") "
Sep 15 20:28:51.293976 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.293916    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28ws7\" (UniqueName: \"kubernetes.io/projected/5a5673a7-326b-441f-9fd6-2a3560e16199-kube-api-access-28ws7\") pod \"5a5673a7-326b-441f-9fd6-2a3560e16199\" (UID: \"5a5673a7-326b-441f-9fd6-2a3560e16199\") "
Sep 15 20:28:51.293976 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.293966    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mdv2v\" (UniqueName: \"kubernetes.io/projected/4fad2e57-3655-4526-a43e-8d57820aba63-kube-api-access-mdv2v\") pod \"4fad2e57-3655-4526-a43e-8d57820aba63\" (UID: \"4fad2e57-3655-4526-a43e-8d57820aba63\") "
Sep 15 20:28:51.294276 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.294246    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fad2e57-3655-4526-a43e-8d57820aba63-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4fad2e57-3655-4526-a43e-8d57820aba63" (UID: "4fad2e57-3655-4526-a43e-8d57820aba63"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:28:51.294276 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.294261    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a5673a7-326b-441f-9fd6-2a3560e16199-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5a5673a7-326b-441f-9fd6-2a3560e16199" (UID: "5a5673a7-326b-441f-9fd6-2a3560e16199"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:28:51.294404 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.294381    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fad2e57-3655-4526-a43e-8d57820aba63-plugins-config-volume\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:28:51.294444 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.294401    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a5673a7-326b-441f-9fd6-2a3560e16199-plugins-config-volume\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:28:51.295881 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.295861    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fad2e57-3655-4526-a43e-8d57820aba63-kube-api-access-mdv2v" (OuterVolumeSpecName: "kube-api-access-mdv2v") pod "4fad2e57-3655-4526-a43e-8d57820aba63" (UID: "4fad2e57-3655-4526-a43e-8d57820aba63"). InnerVolumeSpecName "kube-api-access-mdv2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:28:51.295958 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.295889    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a5673a7-326b-441f-9fd6-2a3560e16199-kube-api-access-28ws7" (OuterVolumeSpecName: "kube-api-access-28ws7") pod "5a5673a7-326b-441f-9fd6-2a3560e16199" (UID: "5a5673a7-326b-441f-9fd6-2a3560e16199"). InnerVolumeSpecName "kube-api-access-28ws7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:28:51.395289 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.395260    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdv2v\" (UniqueName: \"kubernetes.io/projected/4fad2e57-3655-4526-a43e-8d57820aba63-kube-api-access-mdv2v\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:28:51.395289 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.395284    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28ws7\" (UniqueName: \"kubernetes.io/projected/5a5673a7-326b-441f-9fd6-2a3560e16199-kube-api-access-28ws7\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:28:51.428148 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.428097    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:51.550346 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.550319    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:51.576498 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.576473    2576 generic.go:356] "Generic (PLEG): container finished" podID="4fad2e57-3655-4526-a43e-8d57820aba63" containerID="a1a06a9254dd55886965af730907ce8cf7dcb26201f500bde9676797d3af2a70" exitCode=0
Sep 15 20:28:51.576617 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.576513    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-01ea3b-5cfd8d6bdf-ln7n4"
Sep 15 20:28:51.576617 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.576556    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-01ea3b-5cfd8d6bdf-ln7n4" event={"ID":"4fad2e57-3655-4526-a43e-8d57820aba63","Type":"ContainerDied","Data":"a1a06a9254dd55886965af730907ce8cf7dcb26201f500bde9676797d3af2a70"}
Sep 15 20:28:51.576617 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.576599    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-01ea3b-5cfd8d6bdf-ln7n4" event={"ID":"4fad2e57-3655-4526-a43e-8d57820aba63","Type":"ContainerDied","Data":"70edb8568a11fb44b9daf820686edc7540f7f3bc21a5977cf1362d5859df902d"}
Sep 15 20:28:51.576758 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.576621    2576 scope.go:117] "RemoveContainer" containerID="a1a06a9254dd55886965af730907ce8cf7dcb26201f500bde9676797d3af2a70"
Sep 15 20:28:51.578034 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.578011    2576 generic.go:356] "Generic (PLEG): container finished" podID="5a5673a7-326b-441f-9fd6-2a3560e16199" containerID="99d76e9629e25b0c841cf64dca7f97fd2b30045fa56d40f50aaa53cf14872600" exitCode=0
Sep 15 20:28:51.578156 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.578051    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-01ea3b-7c64f65746-h5cxj"
Sep 15 20:28:51.578156 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.578115    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-01ea3b-7c64f65746-h5cxj" event={"ID":"5a5673a7-326b-441f-9fd6-2a3560e16199","Type":"ContainerDied","Data":"99d76e9629e25b0c841cf64dca7f97fd2b30045fa56d40f50aaa53cf14872600"}
Sep 15 20:28:51.578255 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.578163    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-01ea3b-7c64f65746-h5cxj" event={"ID":"5a5673a7-326b-441f-9fd6-2a3560e16199","Type":"ContainerDied","Data":"c5c8a273d3a2683ee6a580e3662f85b571ec993c9f5d7b79cf5431570f9e82dc"}
Sep 15 20:28:51.586406 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.586391    2576 scope.go:117] "RemoveContainer" containerID="a1a06a9254dd55886965af730907ce8cf7dcb26201f500bde9676797d3af2a70"
Sep 15 20:28:51.586731 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:28:51.586698    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1a06a9254dd55886965af730907ce8cf7dcb26201f500bde9676797d3af2a70\": container with ID starting with a1a06a9254dd55886965af730907ce8cf7dcb26201f500bde9676797d3af2a70 not found: ID does not exist" containerID="a1a06a9254dd55886965af730907ce8cf7dcb26201f500bde9676797d3af2a70"
Sep 15 20:28:51.586826 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.586736    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1a06a9254dd55886965af730907ce8cf7dcb26201f500bde9676797d3af2a70"} err="failed to get container status \"a1a06a9254dd55886965af730907ce8cf7dcb26201f500bde9676797d3af2a70\": rpc error: code = NotFound desc = could not find container \"a1a06a9254dd55886965af730907ce8cf7dcb26201f500bde9676797d3af2a70\": container with ID starting with a1a06a9254dd55886965af730907ce8cf7dcb26201f500bde9676797d3af2a70 not found: ID does not exist"
Sep 15 20:28:51.586826 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.586757    2576 scope.go:117] "RemoveContainer" containerID="99d76e9629e25b0c841cf64dca7f97fd2b30045fa56d40f50aaa53cf14872600"
Sep 15 20:28:51.594990 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.594975    2576 scope.go:117] "RemoveContainer" containerID="99d76e9629e25b0c841cf64dca7f97fd2b30045fa56d40f50aaa53cf14872600"
Sep 15 20:28:51.595230 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:28:51.595211    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99d76e9629e25b0c841cf64dca7f97fd2b30045fa56d40f50aaa53cf14872600\": container with ID starting with 99d76e9629e25b0c841cf64dca7f97fd2b30045fa56d40f50aaa53cf14872600 not found: ID does not exist" containerID="99d76e9629e25b0c841cf64dca7f97fd2b30045fa56d40f50aaa53cf14872600"
Sep 15 20:28:51.595277 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.595237    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99d76e9629e25b0c841cf64dca7f97fd2b30045fa56d40f50aaa53cf14872600"} err="failed to get container status \"99d76e9629e25b0c841cf64dca7f97fd2b30045fa56d40f50aaa53cf14872600\": rpc error: code = NotFound desc = could not find container \"99d76e9629e25b0c841cf64dca7f97fd2b30045fa56d40f50aaa53cf14872600\": container with ID starting with 99d76e9629e25b0c841cf64dca7f97fd2b30045fa56d40f50aaa53cf14872600 not found: ID does not exist"
Sep 15 20:28:51.601583 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.601560    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-01ea3b-7c64f65746-h5cxj"]
Sep 15 20:28:51.604337 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.604317    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-01ea3b-7c64f65746-h5cxj"]
Sep 15 20:28:51.614333 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.614313    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-01ea3b-5cfd8d6bdf-ln7n4"]
Sep 15 20:28:51.619408 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:51.619388    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-01ea3b-5cfd8d6bdf-ln7n4"]
Sep 15 20:28:52.143847 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:52.143816    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4fad2e57-3655-4526-a43e-8d57820aba63" path="/var/lib/kubelet/pods/4fad2e57-3655-4526-a43e-8d57820aba63/volumes"
Sep 15 20:28:52.144219 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:52.144206    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a5673a7-326b-441f-9fd6-2a3560e16199" path="/var/lib/kubelet/pods/5a5673a7-326b-441f-9fd6-2a3560e16199/volumes"
Sep 15 20:28:52.201019 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:52.200943    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:52.924495 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:52.924460    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:53.771051 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:53.771019    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:54.483702 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:54.483665    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:55.180277 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:55.180238    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t" podUID="60b9e08a-25f4-4c0b-94a1-eabefde3b249" containerName="main" containerID="cri-o://99a074b3e4f297fd0dd7b0b99a4dbae604dc2eefb45a9e0a205192c2cf6e1a5a" gracePeriod=45
Sep 15 20:28:55.220639 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:55.220606    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:55.443228 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:55.443173    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t"
Sep 15 20:28:55.535747 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:55.535716    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60b9e08a-25f4-4c0b-94a1-eabefde3b249-home\") pod \"60b9e08a-25f4-4c0b-94a1-eabefde3b249\" (UID: \"60b9e08a-25f4-4c0b-94a1-eabefde3b249\") "
Sep 15 20:28:55.535964 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:55.535760    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60b9e08a-25f4-4c0b-94a1-eabefde3b249-dshm\") pod \"60b9e08a-25f4-4c0b-94a1-eabefde3b249\" (UID: \"60b9e08a-25f4-4c0b-94a1-eabefde3b249\") "
Sep 15 20:28:55.535964 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:55.535830    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hw48v\" (UniqueName: \"kubernetes.io/projected/60b9e08a-25f4-4c0b-94a1-eabefde3b249-kube-api-access-hw48v\") pod \"60b9e08a-25f4-4c0b-94a1-eabefde3b249\" (UID: \"60b9e08a-25f4-4c0b-94a1-eabefde3b249\") "
Sep 15 20:28:55.535964 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:55.535850    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60b9e08a-25f4-4c0b-94a1-eabefde3b249-model-cache\") pod \"60b9e08a-25f4-4c0b-94a1-eabefde3b249\" (UID: \"60b9e08a-25f4-4c0b-94a1-eabefde3b249\") "
Sep 15 20:28:55.535964 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:55.535866    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60b9e08a-25f4-4c0b-94a1-eabefde3b249-tmp-dir\") pod \"60b9e08a-25f4-4c0b-94a1-eabefde3b249\" (UID: \"60b9e08a-25f4-4c0b-94a1-eabefde3b249\") "
Sep 15 20:28:55.535964 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:55.535881    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60b9e08a-25f4-4c0b-94a1-eabefde3b249-tls-certs\") pod \"60b9e08a-25f4-4c0b-94a1-eabefde3b249\" (UID: \"60b9e08a-25f4-4c0b-94a1-eabefde3b249\") "
Sep 15 20:28:55.536251 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:55.536043    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60b9e08a-25f4-4c0b-94a1-eabefde3b249-home" (OuterVolumeSpecName: "home") pod "60b9e08a-25f4-4c0b-94a1-eabefde3b249" (UID: "60b9e08a-25f4-4c0b-94a1-eabefde3b249"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:28:55.536251 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:55.536086    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60b9e08a-25f4-4c0b-94a1-eabefde3b249-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "60b9e08a-25f4-4c0b-94a1-eabefde3b249" (UID: "60b9e08a-25f4-4c0b-94a1-eabefde3b249"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:28:55.536251 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:55.536129    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60b9e08a-25f4-4c0b-94a1-eabefde3b249-model-cache" (OuterVolumeSpecName: "model-cache") pod "60b9e08a-25f4-4c0b-94a1-eabefde3b249" (UID: "60b9e08a-25f4-4c0b-94a1-eabefde3b249"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:28:55.536356 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:55.536277    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60b9e08a-25f4-4c0b-94a1-eabefde3b249-home\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:28:55.536356 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:55.536291    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60b9e08a-25f4-4c0b-94a1-eabefde3b249-model-cache\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:28:55.536356 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:55.536301    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60b9e08a-25f4-4c0b-94a1-eabefde3b249-tmp-dir\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:28:55.538192 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:55.538170    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60b9e08a-25f4-4c0b-94a1-eabefde3b249-kube-api-access-hw48v" (OuterVolumeSpecName: "kube-api-access-hw48v") pod "60b9e08a-25f4-4c0b-94a1-eabefde3b249" (UID: "60b9e08a-25f4-4c0b-94a1-eabefde3b249"). InnerVolumeSpecName "kube-api-access-hw48v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:28:55.538272 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:55.538212    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60b9e08a-25f4-4c0b-94a1-eabefde3b249-dshm" (OuterVolumeSpecName: "dshm") pod "60b9e08a-25f4-4c0b-94a1-eabefde3b249" (UID: "60b9e08a-25f4-4c0b-94a1-eabefde3b249"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:28:55.538272 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:55.538255    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60b9e08a-25f4-4c0b-94a1-eabefde3b249-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "60b9e08a-25f4-4c0b-94a1-eabefde3b249" (UID: "60b9e08a-25f4-4c0b-94a1-eabefde3b249"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:28:55.605906 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:55.605877    2576 generic.go:356] "Generic (PLEG): container finished" podID="60b9e08a-25f4-4c0b-94a1-eabefde3b249" containerID="99a074b3e4f297fd0dd7b0b99a4dbae604dc2eefb45a9e0a205192c2cf6e1a5a" exitCode=0
Sep 15 20:28:55.606059 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:55.605950    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t"
Sep 15 20:28:55.606059 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:55.605963    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t" event={"ID":"60b9e08a-25f4-4c0b-94a1-eabefde3b249","Type":"ContainerDied","Data":"99a074b3e4f297fd0dd7b0b99a4dbae604dc2eefb45a9e0a205192c2cf6e1a5a"}
Sep 15 20:28:55.606059 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:55.606003    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t" event={"ID":"60b9e08a-25f4-4c0b-94a1-eabefde3b249","Type":"ContainerDied","Data":"1552c7808af0a07c11b4ccf47557eb76e08bff6efbbdb1a8aa45ccefd7dc15ce"}
Sep 15 20:28:55.606059 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:55.606021    2576 scope.go:117] "RemoveContainer" containerID="99a074b3e4f297fd0dd7b0b99a4dbae604dc2eefb45a9e0a205192c2cf6e1a5a"
Sep 15 20:28:55.615348 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:55.615330    2576 scope.go:117] "RemoveContainer" containerID="99a074b3e4f297fd0dd7b0b99a4dbae604dc2eefb45a9e0a205192c2cf6e1a5a"
Sep 15 20:28:55.615623 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:28:55.615606    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99a074b3e4f297fd0dd7b0b99a4dbae604dc2eefb45a9e0a205192c2cf6e1a5a\": container with ID starting with 99a074b3e4f297fd0dd7b0b99a4dbae604dc2eefb45a9e0a205192c2cf6e1a5a not found: ID does not exist" containerID="99a074b3e4f297fd0dd7b0b99a4dbae604dc2eefb45a9e0a205192c2cf6e1a5a"
Sep 15 20:28:55.615672 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:55.615631    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99a074b3e4f297fd0dd7b0b99a4dbae604dc2eefb45a9e0a205192c2cf6e1a5a"} err="failed to get container status \"99a074b3e4f297fd0dd7b0b99a4dbae604dc2eefb45a9e0a205192c2cf6e1a5a\": rpc error: code = NotFound desc = could not find container \"99a074b3e4f297fd0dd7b0b99a4dbae604dc2eefb45a9e0a205192c2cf6e1a5a\": container with ID starting with 99a074b3e4f297fd0dd7b0b99a4dbae604dc2eefb45a9e0a205192c2cf6e1a5a not found: ID does not exist"
Sep 15 20:28:55.626374 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:55.626350    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t"]
Sep 15 20:28:55.630445 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:55.630426    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-39c7b0/test-model-39c7b0-kserve-75cbfddd57-wx97t"]
Sep 15 20:28:55.637646 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:55.637631    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60b9e08a-25f4-4c0b-94a1-eabefde3b249-dshm\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:28:55.637709 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:55.637649    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hw48v\" (UniqueName: \"kubernetes.io/projected/60b9e08a-25f4-4c0b-94a1-eabefde3b249-kube-api-access-hw48v\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:28:55.637709 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:55.637661    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60b9e08a-25f4-4c0b-94a1-eabefde3b249-tls-certs\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:28:55.963832 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:55.963799    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:56.144411 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:56.144380    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60b9e08a-25f4-4c0b-94a1-eabefde3b249" path="/var/lib/kubelet/pods/60b9e08a-25f4-4c0b-94a1-eabefde3b249/volumes"
Sep 15 20:28:57.292512 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:57.292475    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:57.365275 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:57.365250    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:57.589079 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:57.588999    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:57.894777 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:57.894703    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-39c7b0-p8rfs"]
Sep 15 20:28:57.895170 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:57.895156    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fad2e57-3655-4526-a43e-8d57820aba63" containerName="payload-processing"
Sep 15 20:28:57.895218 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:57.895172    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fad2e57-3655-4526-a43e-8d57820aba63" containerName="payload-processing"
Sep 15 20:28:57.895218 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:57.895184    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6aa92f4-1431-45b1-85a6-29a91e70acdf" containerName="main"
Sep 15 20:28:57.895218 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:57.895190    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6aa92f4-1431-45b1-85a6-29a91e70acdf" containerName="main"
Sep 15 20:28:57.895218 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:57.895200    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60b9e08a-25f4-4c0b-94a1-eabefde3b249" containerName="main"
Sep 15 20:28:57.895218 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:57.895206    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="60b9e08a-25f4-4c0b-94a1-eabefde3b249" containerName="main"
Sep 15 20:28:57.895218 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:57.895213    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a5673a7-326b-441f-9fd6-2a3560e16199" containerName="payload-pre-processing"
Sep 15 20:28:57.895218 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:57.895218    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a5673a7-326b-441f-9fd6-2a3560e16199" containerName="payload-pre-processing"
Sep 15 20:28:57.895439 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:57.895280    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fad2e57-3655-4526-a43e-8d57820aba63" containerName="payload-processing"
Sep 15 20:28:57.895439 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:57.895288    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="60b9e08a-25f4-4c0b-94a1-eabefde3b249" containerName="main"
Sep 15 20:28:57.895439 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:57.895294    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6aa92f4-1431-45b1-85a6-29a91e70acdf" containerName="main"
Sep 15 20:28:57.895439 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:57.895302    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a5673a7-326b-441f-9fd6-2a3560e16199" containerName="payload-pre-processing"
Sep 15 20:28:57.899839 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:57.899823    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-39c7b0-p8rfs"
Sep 15 20:28:57.902139 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:57.902113    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r5lpt\""
Sep 15 20:28:57.902139 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:57.902115    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 20:28:57.906894 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:57.906868    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-39c7b0-p8rfs"]
Sep 15 20:28:57.958358 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:57.958323    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc48fffb-3121-4502-a7d1-c1352613684a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-39c7b0-p8rfs\" (UID: \"cc48fffb-3121-4502-a7d1-c1352613684a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-39c7b0-p8rfs"
Sep 15 20:28:58.059352 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:58.059316    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc48fffb-3121-4502-a7d1-c1352613684a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-39c7b0-p8rfs\" (UID: \"cc48fffb-3121-4502-a7d1-c1352613684a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-39c7b0-p8rfs"
Sep 15 20:28:58.059984 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:58.059965    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc48fffb-3121-4502-a7d1-c1352613684a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-39c7b0-p8rfs\" (UID: \"cc48fffb-3121-4502-a7d1-c1352613684a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-39c7b0-p8rfs"
Sep 15 20:28:58.163654 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:58.163556    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:58.213022 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:58.212996    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-39c7b0-p8rfs"
Sep 15 20:28:58.484452 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:58.484429    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-39c7b0-p8rfs"]
Sep 15 20:28:58.486481 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:28:58.486455    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc48fffb_3121_4502_a7d1_c1352613684a.slice/crio-89b394162304abe288285aabd764cac71258be38d57b265fc7b6bd6da43f077a WatchSource:0}: Error finding container 89b394162304abe288285aabd764cac71258be38d57b265fc7b6bd6da43f077a: Status 404 returned error can't find the container with id 89b394162304abe288285aabd764cac71258be38d57b265fc7b6bd6da43f077a
Sep 15 20:28:58.488502 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:58.488484    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 20:28:58.625617 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:58.625588    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-39c7b0-p8rfs" event={"ID":"cc48fffb-3121-4502-a7d1-c1352613684a","Type":"ContainerStarted","Data":"89b394162304abe288285aabd764cac71258be38d57b265fc7b6bd6da43f077a"}
Sep 15 20:28:58.885331 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:58.885225    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:28:59.657344 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:28:59.657303    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:00.369076 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:00.369039    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:00.536518 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:00.536482    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh"]
Sep 15 20:29:00.546870 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:00.546842    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh"
Sep 15 20:29:00.549340 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:00.549315    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-7e3543\"/\"distinct-2-w0-7e3543-kserve-self-signed-certs\""
Sep 15 20:29:00.549485 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:00.549445    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh"]
Sep 15 20:29:00.650032 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:00.649953    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:00.687323 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:00.687291    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2cd2c1ab-b253-40fd-b902-5de5943ab450-dshm\") pod \"distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh\" (UID: \"2cd2c1ab-b253-40fd-b902-5de5943ab450\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh"
Sep 15 20:29:00.687711 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:00.687359    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2cd2c1ab-b253-40fd-b902-5de5943ab450-home\") pod \"distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh\" (UID: \"2cd2c1ab-b253-40fd-b902-5de5943ab450\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh"
Sep 15 20:29:00.687711 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:00.687426    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2cd2c1ab-b253-40fd-b902-5de5943ab450-model-cache\") pod \"distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh\" (UID: \"2cd2c1ab-b253-40fd-b902-5de5943ab450\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh"
Sep 15 20:29:00.687711 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:00.687473    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2cd2c1ab-b253-40fd-b902-5de5943ab450-tls-certs\") pod \"distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh\" (UID: \"2cd2c1ab-b253-40fd-b902-5de5943ab450\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh"
Sep 15 20:29:00.687711 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:00.687579    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpkbc\" (UniqueName: \"kubernetes.io/projected/2cd2c1ab-b253-40fd-b902-5de5943ab450-kube-api-access-cpkbc\") pod \"distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh\" (UID: \"2cd2c1ab-b253-40fd-b902-5de5943ab450\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh"
Sep 15 20:29:00.687711 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:00.687705    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2cd2c1ab-b253-40fd-b902-5de5943ab450-tmp-dir\") pod \"distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh\" (UID: \"2cd2c1ab-b253-40fd-b902-5de5943ab450\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh"
Sep 15 20:29:00.788774 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:00.788740    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2cd2c1ab-b253-40fd-b902-5de5943ab450-tmp-dir\") pod \"distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh\" (UID: \"2cd2c1ab-b253-40fd-b902-5de5943ab450\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh"
Sep 15 20:29:00.788943 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:00.788797    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2cd2c1ab-b253-40fd-b902-5de5943ab450-dshm\") pod \"distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh\" (UID: \"2cd2c1ab-b253-40fd-b902-5de5943ab450\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh"
Sep 15 20:29:00.788943 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:00.788832    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2cd2c1ab-b253-40fd-b902-5de5943ab450-home\") pod \"distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh\" (UID: \"2cd2c1ab-b253-40fd-b902-5de5943ab450\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh"
Sep 15 20:29:00.788943 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:00.788855    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2cd2c1ab-b253-40fd-b902-5de5943ab450-model-cache\") pod \"distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh\" (UID: \"2cd2c1ab-b253-40fd-b902-5de5943ab450\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh"
Sep 15 20:29:00.788943 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:00.788878    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2cd2c1ab-b253-40fd-b902-5de5943ab450-tls-certs\") pod \"distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh\" (UID: \"2cd2c1ab-b253-40fd-b902-5de5943ab450\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh"
Sep 15 20:29:00.789145 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:00.788989    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpkbc\" (UniqueName: \"kubernetes.io/projected/2cd2c1ab-b253-40fd-b902-5de5943ab450-kube-api-access-cpkbc\") pod \"distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh\" (UID: \"2cd2c1ab-b253-40fd-b902-5de5943ab450\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh"
Sep 15 20:29:00.789232 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:00.789210    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2cd2c1ab-b253-40fd-b902-5de5943ab450-tmp-dir\") pod \"distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh\" (UID: \"2cd2c1ab-b253-40fd-b902-5de5943ab450\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh"
Sep 15 20:29:00.789288 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:00.789240    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2cd2c1ab-b253-40fd-b902-5de5943ab450-model-cache\") pod \"distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh\" (UID: \"2cd2c1ab-b253-40fd-b902-5de5943ab450\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh"
Sep 15 20:29:00.789288 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:00.789269    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2cd2c1ab-b253-40fd-b902-5de5943ab450-home\") pod \"distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh\" (UID: \"2cd2c1ab-b253-40fd-b902-5de5943ab450\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh"
Sep 15 20:29:00.791365 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:00.791342    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2cd2c1ab-b253-40fd-b902-5de5943ab450-tls-certs\") pod \"distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh\" (UID: \"2cd2c1ab-b253-40fd-b902-5de5943ab450\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh"
Sep 15 20:29:00.791365 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:00.791359    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2cd2c1ab-b253-40fd-b902-5de5943ab450-dshm\") pod \"distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh\" (UID: \"2cd2c1ab-b253-40fd-b902-5de5943ab450\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh"
Sep 15 20:29:00.797119 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:00.797093    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpkbc\" (UniqueName: \"kubernetes.io/projected/2cd2c1ab-b253-40fd-b902-5de5943ab450-kube-api-access-cpkbc\") pod \"distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh\" (UID: \"2cd2c1ab-b253-40fd-b902-5de5943ab450\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh"
Sep 15 20:29:00.860593 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:00.860566    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh"
Sep 15 20:29:01.133731 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:01.133708    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh"]
Sep 15 20:29:01.135391 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:29:01.135370    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2cd2c1ab_b253_40fd_b902_5de5943ab450.slice/crio-7f1784cac342d4e736ccd1501ea3889f9508be5cc4cb9d78750d37b0bcd1209b WatchSource:0}: Error finding container 7f1784cac342d4e736ccd1501ea3889f9508be5cc4cb9d78750d37b0bcd1209b: Status 404 returned error can't find the container with id 7f1784cac342d4e736ccd1501ea3889f9508be5cc4cb9d78750d37b0bcd1209b
Sep 15 20:29:01.160916 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:01.160846    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:01.645231 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:01.645189    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh" event={"ID":"2cd2c1ab-b253-40fd-b902-5de5943ab450","Type":"ContainerStarted","Data":"63ebcb390653f52a02e5329c8b180fc25f0d5fb9214b3b64a1fb8b89ef2c0900"}
Sep 15 20:29:01.645380 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:01.645244    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh" event={"ID":"2cd2c1ab-b253-40fd-b902-5de5943ab450","Type":"ContainerStarted","Data":"7f1784cac342d4e736ccd1501ea3889f9508be5cc4cb9d78750d37b0bcd1209b"}
Sep 15 20:29:01.646763 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:01.646741    2576 generic.go:356] "Generic (PLEG): container finished" podID="cc48fffb-3121-4502-a7d1-c1352613684a" containerID="9380fb6dbb38c7c4153a9f4135fe4e5cff06cc905d61f8de5a1e5774315fcd10" exitCode=0
Sep 15 20:29:01.646847 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:01.646776    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-39c7b0-p8rfs" event={"ID":"cc48fffb-3121-4502-a7d1-c1352613684a","Type":"ContainerDied","Data":"9380fb6dbb38c7c4153a9f4135fe4e5cff06cc905d61f8de5a1e5774315fcd10"}
Sep 15 20:29:01.662869 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:01.662821    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh" podStartSLOduration=1.66280625 podStartE2EDuration="1.66280625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:29:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:29:01.660298918 +0000 UTC m=+1456.148763405" watchObservedRunningTime="2026-09-15 20:29:01.66280625 +0000 UTC m=+1456.151270725"
Sep 15 20:29:01.942224 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:01.942195    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:02.658868 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:02.658837    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:02.793303 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:02.793278    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-39c7b0-p8rfs"
Sep 15 20:29:02.910888 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:02.910801    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc48fffb-3121-4502-a7d1-c1352613684a-maas-api-service-ca\") pod \"cc48fffb-3121-4502-a7d1-c1352613684a\" (UID: \"cc48fffb-3121-4502-a7d1-c1352613684a\") "
Sep 15 20:29:02.911191 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:02.911170    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc48fffb-3121-4502-a7d1-c1352613684a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cc48fffb-3121-4502-a7d1-c1352613684a" (UID: "cc48fffb-3121-4502-a7d1-c1352613684a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:29:03.011887 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:03.011848    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc48fffb-3121-4502-a7d1-c1352613684a-maas-api-service-ca\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:29:03.331747 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:03.331690    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:03.411709 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:03.411678    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz"]
Sep 15 20:29:03.443614 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:03.443570    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:03.526519 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:03.526491    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:03.658831 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:03.658745    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-39c7b0-p8rfs"
Sep 15 20:29:03.658831 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:03.658762    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-39c7b0-p8rfs" event={"ID":"cc48fffb-3121-4502-a7d1-c1352613684a","Type":"ContainerDied","Data":"89b394162304abe288285aabd764cac71258be38d57b265fc7b6bd6da43f077a"}
Sep 15 20:29:03.658831 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:03.658813    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="89b394162304abe288285aabd764cac71258be38d57b265fc7b6bd6da43f077a"
Sep 15 20:29:03.695702 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:03.695668    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:03.798941 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:03.798890    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:08.537886 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:08.537850    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:08.567191 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:08.567157    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-39c7b0-56db6464b6-6tgv8"]
Sep 15 20:29:08.567506 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:08.567445    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39c7b0-56db6464b6-6tgv8" podUID="90ad8ae4-67bf-49ae-bfcb-19d551508307" containerName="payload-pre-processing" containerID="cri-o://f6c20aaadcdc77bea405895e0e539bd2f6ceaec46ca71625a518d27faee29ff2" gracePeriod=30
Sep 15 20:29:08.603045 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:08.603015    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:08.686768 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:08.686741    2576 generic.go:356] "Generic (PLEG): container finished" podID="90ad8ae4-67bf-49ae-bfcb-19d551508307" containerID="f6c20aaadcdc77bea405895e0e539bd2f6ceaec46ca71625a518d27faee29ff2" exitCode=0
Sep 15 20:29:08.686899 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:08.686810    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39c7b0-56db6464b6-6tgv8" event={"ID":"90ad8ae4-67bf-49ae-bfcb-19d551508307","Type":"ContainerDied","Data":"f6c20aaadcdc77bea405895e0e539bd2f6ceaec46ca71625a518d27faee29ff2"}
Sep 15 20:29:08.704722 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:08.704693    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:08.741765 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:08.741722    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-39c7b0-p8rfs"]
Sep 15 20:29:08.745012 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:08.744984    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-39c7b0-p8rfs"]
Sep 15 20:29:08.761115 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:08.761062    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:08.817500 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:08.817481    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39c7b0-56db6464b6-6tgv8"
Sep 15 20:29:08.827008 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:08.826982    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:08.969110 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:08.969082    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5x6dz\" (UniqueName: \"kubernetes.io/projected/90ad8ae4-67bf-49ae-bfcb-19d551508307-kube-api-access-5x6dz\") pod \"90ad8ae4-67bf-49ae-bfcb-19d551508307\" (UID: \"90ad8ae4-67bf-49ae-bfcb-19d551508307\") "
Sep 15 20:29:08.969256 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:08.969197    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90ad8ae4-67bf-49ae-bfcb-19d551508307-plugins-config-volume\") pod \"90ad8ae4-67bf-49ae-bfcb-19d551508307\" (UID: \"90ad8ae4-67bf-49ae-bfcb-19d551508307\") "
Sep 15 20:29:08.969562 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:08.969542    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90ad8ae4-67bf-49ae-bfcb-19d551508307-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "90ad8ae4-67bf-49ae-bfcb-19d551508307" (UID: "90ad8ae4-67bf-49ae-bfcb-19d551508307"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:29:08.971218 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:08.971193    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90ad8ae4-67bf-49ae-bfcb-19d551508307-kube-api-access-5x6dz" (OuterVolumeSpecName: "kube-api-access-5x6dz") pod "90ad8ae4-67bf-49ae-bfcb-19d551508307" (UID: "90ad8ae4-67bf-49ae-bfcb-19d551508307"). InnerVolumeSpecName "kube-api-access-5x6dz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:29:09.070385 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:09.070324    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90ad8ae4-67bf-49ae-bfcb-19d551508307-plugins-config-volume\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:29:09.070385 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:09.070349    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5x6dz\" (UniqueName: \"kubernetes.io/projected/90ad8ae4-67bf-49ae-bfcb-19d551508307-kube-api-access-5x6dz\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:29:09.118441 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:09.118414    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:09.222494 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:09.222466    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:09.694001 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:09.693968    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39c7b0-56db6464b6-6tgv8"
Sep 15 20:29:09.694401 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:09.693978    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-39c7b0-56db6464b6-6tgv8" event={"ID":"90ad8ae4-67bf-49ae-bfcb-19d551508307","Type":"ContainerDied","Data":"5f0541dcc4fe0b5531084de2f6e67c7b32aa2f24ac1df769525553ffbdee5d10"}
Sep 15 20:29:09.694401 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:09.694097    2576 scope.go:117] "RemoveContainer" containerID="f6c20aaadcdc77bea405895e0e539bd2f6ceaec46ca71625a518d27faee29ff2"
Sep 15 20:29:09.716627 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:09.716565    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-39c7b0-56db6464b6-6tgv8"]
Sep 15 20:29:09.718538 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:09.718518    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-39c7b0-56db6464b6-6tgv8"]
Sep 15 20:29:09.891288 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:09.891254    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:10.143472 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:10.143439    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90ad8ae4-67bf-49ae-bfcb-19d551508307" path="/var/lib/kubelet/pods/90ad8ae4-67bf-49ae-bfcb-19d551508307/volumes"
Sep 15 20:29:10.143800 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:10.143787    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc48fffb-3121-4502-a7d1-c1352613684a" path="/var/lib/kubelet/pods/cc48fffb-3121-4502-a7d1-c1352613684a/volumes"
Sep 15 20:29:10.556325 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:10.556293    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:10.861621 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:10.861536    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh"
Sep 15 20:29:10.862097 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:10.861841    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh"
Sep 15 20:29:10.874381 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:10.874357    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh"
Sep 15 20:29:11.453844 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:11.453813    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:11.717266 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:11.717243    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh"
Sep 15 20:29:12.184740 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.184651    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:12.235025 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.234999    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h"]
Sep 15 20:29:12.235497 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.235473    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc48fffb-3121-4502-a7d1-c1352613684a" containerName="revoke-keys"
Sep 15 20:29:12.235497 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.235498    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc48fffb-3121-4502-a7d1-c1352613684a" containerName="revoke-keys"
Sep 15 20:29:12.235628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.235510    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90ad8ae4-67bf-49ae-bfcb-19d551508307" containerName="payload-pre-processing"
Sep 15 20:29:12.235628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.235517    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="90ad8ae4-67bf-49ae-bfcb-19d551508307" containerName="payload-pre-processing"
Sep 15 20:29:12.235628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.235589    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc48fffb-3121-4502-a7d1-c1352613684a" containerName="revoke-keys"
Sep 15 20:29:12.235628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.235603    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="90ad8ae4-67bf-49ae-bfcb-19d551508307" containerName="payload-pre-processing"
Sep 15 20:29:12.240431 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.240413    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h"
Sep 15 20:29:12.242974 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.242956    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-025acc06\"/\"openshift-service-ca.crt\""
Sep 15 20:29:12.243068 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.242976    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-025acc06\"/\"kube-root-ca.crt\""
Sep 15 20:29:12.243068 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.243027    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-025acc06\"/\"default-dockercfg-5qnrk\""
Sep 15 20:29:12.243745 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.243725    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-025acc06\"/\"e2e-lifecycle-model-025acc06-kserve-self-signed-certs\""
Sep 15 20:29:12.249556 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.249533    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h"]
Sep 15 20:29:12.332344 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.332307    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:12.400739 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.400706    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-tmp-dir\") pod \"e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h\" (UID: \"822b13dc-6e4a-468e-92ff-70fc7b2a55aa\") " pod="ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h"
Sep 15 20:29:12.400903 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.400765    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-model-cache\") pod \"e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h\" (UID: \"822b13dc-6e4a-468e-92ff-70fc7b2a55aa\") " pod="ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h"
Sep 15 20:29:12.400903 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.400862    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-tls-certs\") pod \"e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h\" (UID: \"822b13dc-6e4a-468e-92ff-70fc7b2a55aa\") " pod="ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h"
Sep 15 20:29:12.400903 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.400889    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j8tx\" (UniqueName: \"kubernetes.io/projected/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-kube-api-access-5j8tx\") pod \"e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h\" (UID: \"822b13dc-6e4a-468e-92ff-70fc7b2a55aa\") " pod="ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h"
Sep 15 20:29:12.401071 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.400914    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-home\") pod \"e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h\" (UID: \"822b13dc-6e4a-468e-92ff-70fc7b2a55aa\") " pod="ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h"
Sep 15 20:29:12.401071 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.400992    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-dshm\") pod \"e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h\" (UID: \"822b13dc-6e4a-468e-92ff-70fc7b2a55aa\") " pod="ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h"
Sep 15 20:29:12.501530 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.501500    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-model-cache\") pod \"e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h\" (UID: \"822b13dc-6e4a-468e-92ff-70fc7b2a55aa\") " pod="ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h"
Sep 15 20:29:12.501676 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.501568    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-tls-certs\") pod \"e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h\" (UID: \"822b13dc-6e4a-468e-92ff-70fc7b2a55aa\") " pod="ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h"
Sep 15 20:29:12.501676 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.501595    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j8tx\" (UniqueName: \"kubernetes.io/projected/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-kube-api-access-5j8tx\") pod \"e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h\" (UID: \"822b13dc-6e4a-468e-92ff-70fc7b2a55aa\") " pod="ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h"
Sep 15 20:29:12.501676 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.501625    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-home\") pod \"e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h\" (UID: \"822b13dc-6e4a-468e-92ff-70fc7b2a55aa\") " pod="ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h"
Sep 15 20:29:12.501676 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.501667    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-dshm\") pod \"e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h\" (UID: \"822b13dc-6e4a-468e-92ff-70fc7b2a55aa\") " pod="ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h"
Sep 15 20:29:12.501893 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.501695    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-tmp-dir\") pod \"e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h\" (UID: \"822b13dc-6e4a-468e-92ff-70fc7b2a55aa\") " pod="ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h"
Sep 15 20:29:12.501982 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.501940    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-model-cache\") pod \"e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h\" (UID: \"822b13dc-6e4a-468e-92ff-70fc7b2a55aa\") " pod="ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h"
Sep 15 20:29:12.502069 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.502048    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-home\") pod \"e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h\" (UID: \"822b13dc-6e4a-468e-92ff-70fc7b2a55aa\") " pod="ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h"
Sep 15 20:29:12.502131 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.502074    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-tmp-dir\") pod \"e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h\" (UID: \"822b13dc-6e4a-468e-92ff-70fc7b2a55aa\") " pod="ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h"
Sep 15 20:29:12.503852 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.503834    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-dshm\") pod \"e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h\" (UID: \"822b13dc-6e4a-468e-92ff-70fc7b2a55aa\") " pod="ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h"
Sep 15 20:29:12.504172 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.504153    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-tls-certs\") pod \"e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h\" (UID: \"822b13dc-6e4a-468e-92ff-70fc7b2a55aa\") " pod="ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h"
Sep 15 20:29:12.510546 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.510524    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j8tx\" (UniqueName: \"kubernetes.io/projected/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-kube-api-access-5j8tx\") pod \"e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h\" (UID: \"822b13dc-6e4a-468e-92ff-70fc7b2a55aa\") " pod="ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h"
Sep 15 20:29:12.551208 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.551167    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h"
Sep 15 20:29:12.824915 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.824891    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h"]
Sep 15 20:29:12.826686 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:29:12.826655    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod822b13dc_6e4a_468e_92ff_70fc7b2a55aa.slice/crio-b7662f21d0164ac76866bee5e0f23f548c5b99688a0e3ea2becdc356b9d7328d WatchSource:0}: Error finding container b7662f21d0164ac76866bee5e0f23f548c5b99688a0e3ea2becdc356b9d7328d: Status 404 returned error can't find the container with id b7662f21d0164ac76866bee5e0f23f548c5b99688a0e3ea2becdc356b9d7328d
Sep 15 20:29:12.996088 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:12.996059    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:13.719906 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:13.719868    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h" event={"ID":"822b13dc-6e4a-468e-92ff-70fc7b2a55aa","Type":"ContainerStarted","Data":"b972e0770a65151a6bddc1d3feab017b461e3f5934b310591352937df5ed1d6d"}
Sep 15 20:29:13.720344 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:13.719915    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h" event={"ID":"822b13dc-6e4a-468e-92ff-70fc7b2a55aa","Type":"ContainerStarted","Data":"b7662f21d0164ac76866bee5e0f23f548c5b99688a0e3ea2becdc356b9d7328d"}
Sep 15 20:29:13.737727 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:13.737676    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h" podStartSLOduration=1.737661797 podStartE2EDuration="1.737661797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:29:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:29:13.736762622 +0000 UTC m=+1468.225227103" watchObservedRunningTime="2026-09-15 20:29:13.737661797 +0000 UTC m=+1468.226126272"
Sep 15 20:29:15.236838 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:15.236798    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:15.335999 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:15.335968    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:15.509129 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:15.509051    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:16.052600 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:16.052566    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:16.389272 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:16.389160    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:16.566048 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:16.566005    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:18.432404 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:18.432357    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz" podUID="354151e6-3aa2-484f-99be-7cb3455045c7" containerName="main" containerID="cri-o://7b124e3801fb63f63657d66f2af9c96e9b04e25a16a4c9af5f998313f9f3fcb4" gracePeriod=45
Sep 15 20:29:18.680413 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:18.680388    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz"
Sep 15 20:29:18.747341 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:18.747310    2576 generic.go:356] "Generic (PLEG): container finished" podID="354151e6-3aa2-484f-99be-7cb3455045c7" containerID="7b124e3801fb63f63657d66f2af9c96e9b04e25a16a4c9af5f998313f9f3fcb4" exitCode=0
Sep 15 20:29:18.747515 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:18.747352    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz"
Sep 15 20:29:18.747515 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:18.747400    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz" event={"ID":"354151e6-3aa2-484f-99be-7cb3455045c7","Type":"ContainerDied","Data":"7b124e3801fb63f63657d66f2af9c96e9b04e25a16a4c9af5f998313f9f3fcb4"}
Sep 15 20:29:18.747515 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:18.747444    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz" event={"ID":"354151e6-3aa2-484f-99be-7cb3455045c7","Type":"ContainerDied","Data":"e1092e4da24acbf5ebb33568f064460ef22ee7b7954735b2a9b57249bb92ae35"}
Sep 15 20:29:18.747515 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:18.747467    2576 scope.go:117] "RemoveContainer" containerID="7b124e3801fb63f63657d66f2af9c96e9b04e25a16a4c9af5f998313f9f3fcb4"
Sep 15 20:29:18.756487 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:18.756468    2576 scope.go:117] "RemoveContainer" containerID="7b124e3801fb63f63657d66f2af9c96e9b04e25a16a4c9af5f998313f9f3fcb4"
Sep 15 20:29:18.756763 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:29:18.756743    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b124e3801fb63f63657d66f2af9c96e9b04e25a16a4c9af5f998313f9f3fcb4\": container with ID starting with 7b124e3801fb63f63657d66f2af9c96e9b04e25a16a4c9af5f998313f9f3fcb4 not found: ID does not exist" containerID="7b124e3801fb63f63657d66f2af9c96e9b04e25a16a4c9af5f998313f9f3fcb4"
Sep 15 20:29:18.756806 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:18.756772    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b124e3801fb63f63657d66f2af9c96e9b04e25a16a4c9af5f998313f9f3fcb4"} err="failed to get container status \"7b124e3801fb63f63657d66f2af9c96e9b04e25a16a4c9af5f998313f9f3fcb4\": rpc error: code = NotFound desc = could not find container \"7b124e3801fb63f63657d66f2af9c96e9b04e25a16a4c9af5f998313f9f3fcb4\": container with ID starting with 7b124e3801fb63f63657d66f2af9c96e9b04e25a16a4c9af5f998313f9f3fcb4 not found: ID does not exist"
Sep 15 20:29:18.782991 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:18.782964    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:18.861950 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:18.861909    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/354151e6-3aa2-484f-99be-7cb3455045c7-dshm\") pod \"354151e6-3aa2-484f-99be-7cb3455045c7\" (UID: \"354151e6-3aa2-484f-99be-7cb3455045c7\") "
Sep 15 20:29:18.862048 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:18.861982    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/354151e6-3aa2-484f-99be-7cb3455045c7-tls-certs\") pod \"354151e6-3aa2-484f-99be-7cb3455045c7\" (UID: \"354151e6-3aa2-484f-99be-7cb3455045c7\") "
Sep 15 20:29:18.862125 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:18.862050    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/354151e6-3aa2-484f-99be-7cb3455045c7-home\") pod \"354151e6-3aa2-484f-99be-7cb3455045c7\" (UID: \"354151e6-3aa2-484f-99be-7cb3455045c7\") "
Sep 15 20:29:18.862125 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:18.862097    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/354151e6-3aa2-484f-99be-7cb3455045c7-model-cache\") pod \"354151e6-3aa2-484f-99be-7cb3455045c7\" (UID: \"354151e6-3aa2-484f-99be-7cb3455045c7\") "
Sep 15 20:29:18.862226 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:18.862160    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/354151e6-3aa2-484f-99be-7cb3455045c7-tmp-dir\") pod \"354151e6-3aa2-484f-99be-7cb3455045c7\" (UID: \"354151e6-3aa2-484f-99be-7cb3455045c7\") "
Sep 15 20:29:18.862287 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:18.862222    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qnql7\" (UniqueName: \"kubernetes.io/projected/354151e6-3aa2-484f-99be-7cb3455045c7-kube-api-access-qnql7\") pod \"354151e6-3aa2-484f-99be-7cb3455045c7\" (UID: \"354151e6-3aa2-484f-99be-7cb3455045c7\") "
Sep 15 20:29:18.862339 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:18.862286    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/354151e6-3aa2-484f-99be-7cb3455045c7-home" (OuterVolumeSpecName: "home") pod "354151e6-3aa2-484f-99be-7cb3455045c7" (UID: "354151e6-3aa2-484f-99be-7cb3455045c7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:29:18.862468 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:18.862404    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/354151e6-3aa2-484f-99be-7cb3455045c7-model-cache" (OuterVolumeSpecName: "model-cache") pod "354151e6-3aa2-484f-99be-7cb3455045c7" (UID: "354151e6-3aa2-484f-99be-7cb3455045c7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:29:18.862525 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:18.862475    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/354151e6-3aa2-484f-99be-7cb3455045c7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "354151e6-3aa2-484f-99be-7cb3455045c7" (UID: "354151e6-3aa2-484f-99be-7cb3455045c7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:29:18.862623 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:18.862606    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/354151e6-3aa2-484f-99be-7cb3455045c7-tmp-dir\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:29:18.862685 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:18.862626    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/354151e6-3aa2-484f-99be-7cb3455045c7-home\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:29:18.862685 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:18.862635    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/354151e6-3aa2-484f-99be-7cb3455045c7-model-cache\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:29:18.864126 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:18.864097    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/354151e6-3aa2-484f-99be-7cb3455045c7-dshm" (OuterVolumeSpecName: "dshm") pod "354151e6-3aa2-484f-99be-7cb3455045c7" (UID: "354151e6-3aa2-484f-99be-7cb3455045c7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:29:18.864222 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:18.864165    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/354151e6-3aa2-484f-99be-7cb3455045c7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "354151e6-3aa2-484f-99be-7cb3455045c7" (UID: "354151e6-3aa2-484f-99be-7cb3455045c7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:29:18.864330 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:18.864312    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/354151e6-3aa2-484f-99be-7cb3455045c7-kube-api-access-qnql7" (OuterVolumeSpecName: "kube-api-access-qnql7") pod "354151e6-3aa2-484f-99be-7cb3455045c7" (UID: "354151e6-3aa2-484f-99be-7cb3455045c7"). InnerVolumeSpecName "kube-api-access-qnql7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:29:18.963801 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:18.963775    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnql7\" (UniqueName: \"kubernetes.io/projected/354151e6-3aa2-484f-99be-7cb3455045c7-kube-api-access-qnql7\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:29:18.963801 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:18.963797    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/354151e6-3aa2-484f-99be-7cb3455045c7-dshm\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:29:18.963972 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:18.963806    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/354151e6-3aa2-484f-99be-7cb3455045c7-tls-certs\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:29:19.073582 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:19.073551    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz"]
Sep 15 20:29:19.086559 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:19.086534    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-6733e9/noten-model-6733e9-kserve-7b9ddfc5d7-l4nbz"]
Sep 15 20:29:19.397447 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:19.397371    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:19.490606 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:19.490572    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:19.693092 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:19.693008    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:20.143790 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:20.143757    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="354151e6-3aa2-484f-99be-7cb3455045c7" path="/var/lib/kubelet/pods/354151e6-3aa2-484f-99be-7cb3455045c7/volumes"
Sep 15 20:29:20.671405 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:20.671368    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:21.569054 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:21.569013    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:22.476033 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:22.475998    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:22.552349 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:22.552321    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h"
Sep 15 20:29:22.552517 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:22.552401    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h"
Sep 15 20:29:22.565016 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:22.564994    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h"
Sep 15 20:29:22.780755 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:22.780676    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h"
Sep 15 20:29:23.287713 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:23.287681    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:24.036411 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:24.036379    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:24.139244 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:24.139212    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:24.461510 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:24.461455    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:24.558323 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:24.558283    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:24.594326 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:24.594296    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-92702a-7865d5fc58-zgbpp"]
Sep 15 20:29:24.594770 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:24.594756    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="354151e6-3aa2-484f-99be-7cb3455045c7" containerName="main"
Sep 15 20:29:24.594810 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:24.594771    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="354151e6-3aa2-484f-99be-7cb3455045c7" containerName="main"
Sep 15 20:29:24.594841 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:24.594836    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="354151e6-3aa2-484f-99be-7cb3455045c7" containerName="main"
Sep 15 20:29:24.599277 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:24.599259    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-92702a-7865d5fc58-zgbpp"
Sep 15 20:29:24.601569 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:24.601540    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-92702a\""
Sep 15 20:29:24.601654 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:24.601573    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-92702a-dockercfg-vwk4w\""
Sep 15 20:29:24.606264 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:24.606244    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-92702a-7865d5fc58-zgbpp"]
Sep 15 20:29:24.616389 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:24.616357    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d56d4079-5452-4e54-82fb-06cf8a4ce399-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-92702a-7865d5fc58-zgbpp\" (UID: \"d56d4079-5452-4e54-82fb-06cf8a4ce399\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-92702a-7865d5fc58-zgbpp"
Sep 15 20:29:24.616487 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:24.616450    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27976\" (UniqueName: \"kubernetes.io/projected/d56d4079-5452-4e54-82fb-06cf8a4ce399-kube-api-access-27976\") pod \"payload-pre-processing-e2e-shared-a-w3-92702a-7865d5fc58-zgbpp\" (UID: \"d56d4079-5452-4e54-82fb-06cf8a4ce399\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-92702a-7865d5fc58-zgbpp"
Sep 15 20:29:24.638588 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:24.638559    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-92702a-d74467ff7-6mtgx"]
Sep 15 20:29:24.642671 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:24.642654    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-92702a-d74467ff7-6mtgx"
Sep 15 20:29:24.653133 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:24.653111    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-92702a-d74467ff7-6mtgx"]
Sep 15 20:29:24.662614 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:24.662594    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:24.717101 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:24.717031    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d56d4079-5452-4e54-82fb-06cf8a4ce399-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-92702a-7865d5fc58-zgbpp\" (UID: \"d56d4079-5452-4e54-82fb-06cf8a4ce399\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-92702a-7865d5fc58-zgbpp"
Sep 15 20:29:24.717101 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:24.717093    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-92702a-d74467ff7-6mtgx\" (UID: \"5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-92702a-d74467ff7-6mtgx"
Sep 15 20:29:24.717325 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:24.717122    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kd79g\" (UniqueName: \"kubernetes.io/projected/5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a-kube-api-access-kd79g\") pod \"payload-processing-e2e-shared-a-w3-92702a-d74467ff7-6mtgx\" (UID: \"5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-92702a-d74467ff7-6mtgx"
Sep 15 20:29:24.717325 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:24.717290    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27976\" (UniqueName: \"kubernetes.io/projected/d56d4079-5452-4e54-82fb-06cf8a4ce399-kube-api-access-27976\") pod \"payload-pre-processing-e2e-shared-a-w3-92702a-7865d5fc58-zgbpp\" (UID: \"d56d4079-5452-4e54-82fb-06cf8a4ce399\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-92702a-7865d5fc58-zgbpp"
Sep 15 20:29:24.717682 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:24.717666    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d56d4079-5452-4e54-82fb-06cf8a4ce399-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-92702a-7865d5fc58-zgbpp\" (UID: \"d56d4079-5452-4e54-82fb-06cf8a4ce399\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-92702a-7865d5fc58-zgbpp"
Sep 15 20:29:24.725212 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:24.725186    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27976\" (UniqueName: \"kubernetes.io/projected/d56d4079-5452-4e54-82fb-06cf8a4ce399-kube-api-access-27976\") pod \"payload-pre-processing-e2e-shared-a-w3-92702a-7865d5fc58-zgbpp\" (UID: \"d56d4079-5452-4e54-82fb-06cf8a4ce399\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-92702a-7865d5fc58-zgbpp"
Sep 15 20:29:24.818680 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:24.818652    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-92702a-d74467ff7-6mtgx\" (UID: \"5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-92702a-d74467ff7-6mtgx"
Sep 15 20:29:24.818807 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:24.818694    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kd79g\" (UniqueName: \"kubernetes.io/projected/5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a-kube-api-access-kd79g\") pod \"payload-processing-e2e-shared-a-w3-92702a-d74467ff7-6mtgx\" (UID: \"5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-92702a-d74467ff7-6mtgx"
Sep 15 20:29:24.819362 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:24.819340    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-92702a-d74467ff7-6mtgx\" (UID: \"5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-92702a-d74467ff7-6mtgx"
Sep 15 20:29:24.825900 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:24.825880    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd79g\" (UniqueName: \"kubernetes.io/projected/5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a-kube-api-access-kd79g\") pod \"payload-processing-e2e-shared-a-w3-92702a-d74467ff7-6mtgx\" (UID: \"5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-92702a-d74467ff7-6mtgx"
Sep 15 20:29:24.910759 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:24.910739    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-92702a-7865d5fc58-zgbpp"
Sep 15 20:29:24.954072 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:24.954036    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-92702a-d74467ff7-6mtgx"
Sep 15 20:29:25.182069 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:25.182038    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-92702a-7865d5fc58-zgbpp"]
Sep 15 20:29:25.183960 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:29:25.183934    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd56d4079_5452_4e54_82fb_06cf8a4ce399.slice/crio-bec99a280350a83c828336a02dd2410759a3e2908d34c4d3cfbda6c4caf8dd71 WatchSource:0}: Error finding container bec99a280350a83c828336a02dd2410759a3e2908d34c4d3cfbda6c4caf8dd71: Status 404 returned error can't find the container with id bec99a280350a83c828336a02dd2410759a3e2908d34c4d3cfbda6c4caf8dd71
Sep 15 20:29:25.235631 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:25.235607    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-92702a-d74467ff7-6mtgx"]
Sep 15 20:29:25.239993 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:29:25.239971    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5aff0db4_123b_4f5b_af5d_6c8ca96e3f0a.slice/crio-5cfb4e26fdc8430153f8f2af0a75ba385b9567481b6d0c4d3bfecdc6eb24c77d WatchSource:0}: Error finding container 5cfb4e26fdc8430153f8f2af0a75ba385b9567481b6d0c4d3bfecdc6eb24c77d: Status 404 returned error can't find the container with id 5cfb4e26fdc8430153f8f2af0a75ba385b9567481b6d0c4d3bfecdc6eb24c77d
Sep 15 20:29:25.787124 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:25.787081    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-92702a-d74467ff7-6mtgx" event={"ID":"5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a","Type":"ContainerStarted","Data":"3a4b8b9d90fc9b0e612d8b66c3c304a6a9cfd0994e1cd63e02b76e650e8f6094"}
Sep 15 20:29:25.787307 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:25.787143    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-92702a-d74467ff7-6mtgx" event={"ID":"5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a","Type":"ContainerStarted","Data":"5cfb4e26fdc8430153f8f2af0a75ba385b9567481b6d0c4d3bfecdc6eb24c77d"}
Sep 15 20:29:25.787307 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:25.787172    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-92702a-d74467ff7-6mtgx"
Sep 15 20:29:25.788630 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:25.788608    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-92702a-7865d5fc58-zgbpp" event={"ID":"d56d4079-5452-4e54-82fb-06cf8a4ce399","Type":"ContainerStarted","Data":"19a5d72aa1af58a22ae4b5993d41961e3b2318137a230d381d627f449139ddea"}
Sep 15 20:29:25.788740 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:25.788652    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-92702a-7865d5fc58-zgbpp" event={"ID":"d56d4079-5452-4e54-82fb-06cf8a4ce399","Type":"ContainerStarted","Data":"bec99a280350a83c828336a02dd2410759a3e2908d34c4d3cfbda6c4caf8dd71"}
Sep 15 20:29:25.788740 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:25.788694    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-92702a-7865d5fc58-zgbpp"
Sep 15 20:29:25.804159 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:25.804115    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-92702a-d74467ff7-6mtgx" podStartSLOduration=1.8041017400000001 podStartE2EDuration="1.80410174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:29:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:29:25.801936639 +0000 UTC m=+1480.290401112" watchObservedRunningTime="2026-09-15 20:29:25.80410174 +0000 UTC m=+1480.292566215"
Sep 15 20:29:25.819275 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:25.819227    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-92702a-7865d5fc58-zgbpp" podStartSLOduration=1.819209742 podStartE2EDuration="1.819209742s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:29:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:29:25.816015052 +0000 UTC m=+1480.304479529" watchObservedRunningTime="2026-09-15 20:29:25.819209742 +0000 UTC m=+1480.307674222"
Sep 15 20:29:26.720578 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:26.720547    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:27.088627 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:27.088553    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:36.795490 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:36.795454    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-92702a-d74467ff7-6mtgx"
Sep 15 20:29:36.795891 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:36.795538    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-92702a-7865d5fc58-zgbpp"
Sep 15 20:29:46.229423 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:46.229391    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:47.070932 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:47.070897    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:47.813118 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:47.813089    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:48.458553 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:48.458519    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:49.318187 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:49.318155    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:49.486390 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:49.486358    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:50.647447 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:50.647408    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:50.867187 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:50.867145    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:53.648390 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:53.648359    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:53.734189 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:53.734163    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:53.837261 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:53.837217    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:54.678589 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:54.678523    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:55.465480 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:55.465447    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:56.430087 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:56.430031    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:57.197145 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:57.197110    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:58.051002 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:58.050960    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:58.781569 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:58.781517    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:58.880493 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:58.880457    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:58.947953 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:58.947896    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:59.223870 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:59.223842    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:29:59.314174 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:29:59.314137    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:30:00.144103 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:00.144070    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7e3543-29825070-mkmrz"]
Sep 15 20:30:00.153001 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:00.150908    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-dc12d7-29825070-2k4xs"]
Sep 15 20:30:00.153729 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:00.153700    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7e3543-29825070-mkmrz"
Sep 15 20:30:00.160811 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:00.160781    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-725e67-29825070-jlx6q"]
Sep 15 20:30:00.161188 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:00.161163    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r5lpt\""
Sep 15 20:30:00.170059 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:00.170034    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-dc12d7-29825070-2k4xs"
Sep 15 20:30:00.182422 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:00.182131    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7e3543-29825070-mkmrz"]
Sep 15 20:30:00.182422 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:00.182175    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-725e67-29825070-jlx6q"]
Sep 15 20:30:00.182422 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:00.182201    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f49968-29825070-qh7l4"]
Sep 15 20:30:00.187727 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:00.183112    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-725e67-29825070-jlx6q"
Sep 15 20:30:00.196416 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:00.196385    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f49968-29825070-qh7l4"
Sep 15 20:30:00.199279 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:00.199255    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7e3543-29825070-mkmrz"
Sep 15 20:30:00.199504 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:00.199476    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f49968-29825070-qh7l4"]
Sep 15 20:30:00.206135 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:00.206109    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-dc12d7-29825070-2k4xs"
Sep 15 20:30:00.218547 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:00.218523    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-dc12d7-29825070-2k4xs"]
Sep 15 20:30:00.249558 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:00.249526    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-725e67-29825070-jlx6q"
Sep 15 20:30:00.256639 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:00.256605    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f49968-29825070-qh7l4"
Sep 15 20:30:00.485852 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:00.485730    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7e3543-29825070-mkmrz"]
Sep 15 20:30:00.488642 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:30:00.488601    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11bd406d_e791_4bb2_950b_93b944a22327.slice/crio-2fe993eb314b0e5fea3d345d4553e09b215f2c9bf24060bfcfbd61c904050ca2 WatchSource:0}: Error finding container 2fe993eb314b0e5fea3d345d4553e09b215f2c9bf24060bfcfbd61c904050ca2: Status 404 returned error can't find the container with id 2fe993eb314b0e5fea3d345d4553e09b215f2c9bf24060bfcfbd61c904050ca2
Sep 15 20:30:00.496911 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:00.496887    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-dc12d7-29825070-2k4xs"]
Sep 15 20:30:00.498179 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:30:00.498156    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fbab977_092b_41df_ab62_0542e81e4571.slice/crio-35391b133b57f355ef339a57acec64ed653c3ff9994a984179c17399c7a1cd36 WatchSource:0}: Error finding container 35391b133b57f355ef339a57acec64ed653c3ff9994a984179c17399c7a1cd36: Status 404 returned error can't find the container with id 35391b133b57f355ef339a57acec64ed653c3ff9994a984179c17399c7a1cd36
Sep 15 20:30:00.544572 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:00.544550    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-725e67-29825070-jlx6q"]
Sep 15 20:30:00.545941 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:30:00.545901    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02237da6_c778_40ec_84bf_324dea6667a0.slice/crio-16241386c8f30c7d219068882ecf28d73d2db119bd90c944393c6d6e62c3a2b9 WatchSource:0}: Error finding container 16241386c8f30c7d219068882ecf28d73d2db119bd90c944393c6d6e62c3a2b9: Status 404 returned error can't find the container with id 16241386c8f30c7d219068882ecf28d73d2db119bd90c944393c6d6e62c3a2b9
Sep 15 20:30:00.563840 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:00.563788    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f49968-29825070-qh7l4"]
Sep 15 20:30:00.566752 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:30:00.566727    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7302c55_4f1d_4aa3_80a7_0742f0435ec6.slice/crio-8e2fb20d4164a69895763826ec9f92fe49e89971deaf99df5b33af453cb1ae21 WatchSource:0}: Error finding container 8e2fb20d4164a69895763826ec9f92fe49e89971deaf99df5b33af453cb1ae21: Status 404 returned error can't find the container with id 8e2fb20d4164a69895763826ec9f92fe49e89971deaf99df5b33af453cb1ae21
Sep 15 20:30:00.980259 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:00.980217    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-725e67-29825070-jlx6q" event={"ID":"02237da6-c778-40ec-84bf-324dea6667a0","Type":"ContainerStarted","Data":"cca46fb54b7264b8c7f8a65b2a0d26a061f4009bb97d8291ff5cf48491ee0787"}
Sep 15 20:30:00.980430 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:00.980286    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-725e67-29825070-jlx6q" event={"ID":"02237da6-c778-40ec-84bf-324dea6667a0","Type":"ContainerStarted","Data":"16241386c8f30c7d219068882ecf28d73d2db119bd90c944393c6d6e62c3a2b9"}
Sep 15 20:30:00.981844 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:00.981817    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f49968-29825070-qh7l4" event={"ID":"f7302c55-4f1d-4aa3-80a7-0742f0435ec6","Type":"ContainerStarted","Data":"e16ee77b04c81cd67a4150e432da2ffb3559c8c701035eb0c7b70c1320f51951"}
Sep 15 20:30:00.981969 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:00.981864    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f49968-29825070-qh7l4" event={"ID":"f7302c55-4f1d-4aa3-80a7-0742f0435ec6","Type":"ContainerStarted","Data":"8e2fb20d4164a69895763826ec9f92fe49e89971deaf99df5b33af453cb1ae21"}
Sep 15 20:30:00.986454 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:00.986428    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7e3543-29825070-mkmrz" event={"ID":"11bd406d-e791-4bb2-950b-93b944a22327","Type":"ContainerStarted","Data":"727daf89a2e99319d782e28b8d9344aef8d230c8b6639a983502ddb2ca8a2d6e"}
Sep 15 20:30:00.986585 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:00.986472    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7e3543-29825070-mkmrz" event={"ID":"11bd406d-e791-4bb2-950b-93b944a22327","Type":"ContainerStarted","Data":"2fe993eb314b0e5fea3d345d4553e09b215f2c9bf24060bfcfbd61c904050ca2"}
Sep 15 20:30:00.988173 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:00.988147    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-dc12d7-29825070-2k4xs" event={"ID":"3fbab977-092b-41df-ab62-0542e81e4571","Type":"ContainerStarted","Data":"fa9d1479d0b11c1c2a80a2faa476f6966ad4b0a3d8398be61a045e5c2ef79c53"}
Sep 15 20:30:00.988276 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:00.988191    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-dc12d7-29825070-2k4xs" event={"ID":"3fbab977-092b-41df-ab62-0542e81e4571","Type":"ContainerStarted","Data":"35391b133b57f355ef339a57acec64ed653c3ff9994a984179c17399c7a1cd36"}
Sep 15 20:30:00.993999 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:00.993958    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-725e67-29825070-jlx6q" podStartSLOduration=0.993943473 podStartE2EDuration="993.943473ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:30:00.992916138 +0000 UTC m=+1515.481380615" watchObservedRunningTime="2026-09-15 20:30:00.993943473 +0000 UTC m=+1515.482407947"
Sep 15 20:30:01.007727 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:01.007683    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-dc12d7-29825070-2k4xs" podStartSLOduration=1.007670739 podStartE2EDuration="1.007670739s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:30:01.00609181 +0000 UTC m=+1515.494556310" watchObservedRunningTime="2026-09-15 20:30:01.007670739 +0000 UTC m=+1515.496135215"
Sep 15 20:30:01.021981 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:01.021945    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7e3543-29825070-mkmrz" podStartSLOduration=1.021900642 podStartE2EDuration="1.021900642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:30:01.021675437 +0000 UTC m=+1515.510139914" watchObservedRunningTime="2026-09-15 20:30:01.021900642 +0000 UTC m=+1515.510365118"
Sep 15 20:30:01.035356 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:01.035312    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f49968-29825070-qh7l4" podStartSLOduration=1.035295846 podStartE2EDuration="1.035295846s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:30:01.035269927 +0000 UTC m=+1515.523734430" watchObservedRunningTime="2026-09-15 20:30:01.035295846 +0000 UTC m=+1515.523760323"
Sep 15 20:30:01.996695 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:01.996656    2576 generic.go:356] "Generic (PLEG): container finished" podID="02237da6-c778-40ec-84bf-324dea6667a0" containerID="cca46fb54b7264b8c7f8a65b2a0d26a061f4009bb97d8291ff5cf48491ee0787" exitCode=0
Sep 15 20:30:01.997124 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:01.996743    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-725e67-29825070-jlx6q" event={"ID":"02237da6-c778-40ec-84bf-324dea6667a0","Type":"ContainerDied","Data":"cca46fb54b7264b8c7f8a65b2a0d26a061f4009bb97d8291ff5cf48491ee0787"}
Sep 15 20:30:01.998205 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:01.998184    2576 generic.go:356] "Generic (PLEG): container finished" podID="f7302c55-4f1d-4aa3-80a7-0742f0435ec6" containerID="e16ee77b04c81cd67a4150e432da2ffb3559c8c701035eb0c7b70c1320f51951" exitCode=0
Sep 15 20:30:01.998297 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:01.998261    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f49968-29825070-qh7l4" event={"ID":"f7302c55-4f1d-4aa3-80a7-0742f0435ec6","Type":"ContainerDied","Data":"e16ee77b04c81cd67a4150e432da2ffb3559c8c701035eb0c7b70c1320f51951"}
Sep 15 20:30:01.999978 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:01.999855    2576 generic.go:356] "Generic (PLEG): container finished" podID="11bd406d-e791-4bb2-950b-93b944a22327" containerID="727daf89a2e99319d782e28b8d9344aef8d230c8b6639a983502ddb2ca8a2d6e" exitCode=0
Sep 15 20:30:01.999978 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:01.999959    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7e3543-29825070-mkmrz" event={"ID":"11bd406d-e791-4bb2-950b-93b944a22327","Type":"ContainerDied","Data":"727daf89a2e99319d782e28b8d9344aef8d230c8b6639a983502ddb2ca8a2d6e"}
Sep 15 20:30:02.001512 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:02.001493    2576 generic.go:356] "Generic (PLEG): container finished" podID="3fbab977-092b-41df-ab62-0542e81e4571" containerID="fa9d1479d0b11c1c2a80a2faa476f6966ad4b0a3d8398be61a045e5c2ef79c53" exitCode=0
Sep 15 20:30:02.001614 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:02.001518    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-dc12d7-29825070-2k4xs" event={"ID":"3fbab977-092b-41df-ab62-0542e81e4571","Type":"ContainerDied","Data":"fa9d1479d0b11c1c2a80a2faa476f6966ad4b0a3d8398be61a045e5c2ef79c53"}
Sep 15 20:30:03.154070 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:03.154041    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7e3543-29825070-mkmrz"
Sep 15 20:30:03.191396 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:03.191362    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-725e67-29825070-jlx6q"
Sep 15 20:30:03.252057 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:03.252033    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f49968-29825070-qh7l4"
Sep 15 20:30:03.255305 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:03.255286    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-dc12d7-29825070-2k4xs"
Sep 15 20:30:04.015554 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:04.015524    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7e3543-29825070-mkmrz"
Sep 15 20:30:04.015554 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:04.015533    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7e3543-29825070-mkmrz" event={"ID":"11bd406d-e791-4bb2-950b-93b944a22327","Type":"ContainerDied","Data":"2fe993eb314b0e5fea3d345d4553e09b215f2c9bf24060bfcfbd61c904050ca2"}
Sep 15 20:30:04.015792 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:04.015613    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2fe993eb314b0e5fea3d345d4553e09b215f2c9bf24060bfcfbd61c904050ca2"
Sep 15 20:30:04.016950 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:04.016907    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-dc12d7-29825070-2k4xs"
Sep 15 20:30:04.017073 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:04.016946    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-dc12d7-29825070-2k4xs" event={"ID":"3fbab977-092b-41df-ab62-0542e81e4571","Type":"ContainerDied","Data":"35391b133b57f355ef339a57acec64ed653c3ff9994a984179c17399c7a1cd36"}
Sep 15 20:30:04.017073 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:04.016991    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="35391b133b57f355ef339a57acec64ed653c3ff9994a984179c17399c7a1cd36"
Sep 15 20:30:04.018330 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:04.018303    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-725e67-29825070-jlx6q"
Sep 15 20:30:04.018436 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:04.018328    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-725e67-29825070-jlx6q" event={"ID":"02237da6-c778-40ec-84bf-324dea6667a0","Type":"ContainerDied","Data":"16241386c8f30c7d219068882ecf28d73d2db119bd90c944393c6d6e62c3a2b9"}
Sep 15 20:30:04.018436 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:04.018367    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="16241386c8f30c7d219068882ecf28d73d2db119bd90c944393c6d6e62c3a2b9"
Sep 15 20:30:04.019799 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:04.019779    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f49968-29825070-qh7l4"
Sep 15 20:30:04.019799 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:04.019785    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f49968-29825070-qh7l4" event={"ID":"f7302c55-4f1d-4aa3-80a7-0742f0435ec6","Type":"ContainerDied","Data":"8e2fb20d4164a69895763826ec9f92fe49e89971deaf99df5b33af453cb1ae21"}
Sep 15 20:30:04.019971 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:04.019826    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8e2fb20d4164a69895763826ec9f92fe49e89971deaf99df5b33af453cb1ae21"
Sep 15 20:30:20.844210 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:20.844178    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:30:21.670993 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:21.670959    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:30:23.015020 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:23.014987    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:30:23.115665 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:23.115630    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:30:23.975176 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:23.975144    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:30:24.083114 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:24.083083    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:30:24.899745 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:24.899708    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:30:25.673225 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:25.673195    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:30:26.593084 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:26.593052    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:30:27.866644 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:27.866604    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:30:42.186541 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:42.186506    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-025acc06-lw8df"]
Sep 15 20:30:42.186962 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:42.186916    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11bd406d-e791-4bb2-950b-93b944a22327" containerName="cleanup"
Sep 15 20:30:42.186962 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:42.186943    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="11bd406d-e791-4bb2-950b-93b944a22327" containerName="cleanup"
Sep 15 20:30:42.186962 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:42.186951    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7302c55-4f1d-4aa3-80a7-0742f0435ec6" containerName="cleanup"
Sep 15 20:30:42.186962 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:42.186957    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7302c55-4f1d-4aa3-80a7-0742f0435ec6" containerName="cleanup"
Sep 15 20:30:42.187126 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:42.186970    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fbab977-092b-41df-ab62-0542e81e4571" containerName="cleanup"
Sep 15 20:30:42.187126 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:42.186979    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fbab977-092b-41df-ab62-0542e81e4571" containerName="cleanup"
Sep 15 20:30:42.187126 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:42.187007    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02237da6-c778-40ec-84bf-324dea6667a0" containerName="cleanup"
Sep 15 20:30:42.187126 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:42.187014    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="02237da6-c778-40ec-84bf-324dea6667a0" containerName="cleanup"
Sep 15 20:30:42.187126 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:42.187081    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="02237da6-c778-40ec-84bf-324dea6667a0" containerName="cleanup"
Sep 15 20:30:42.187126 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:42.187089    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7302c55-4f1d-4aa3-80a7-0742f0435ec6" containerName="cleanup"
Sep 15 20:30:42.187126 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:42.187095    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fbab977-092b-41df-ab62-0542e81e4571" containerName="cleanup"
Sep 15 20:30:42.187126 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:42.187103    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="11bd406d-e791-4bb2-950b-93b944a22327" containerName="cleanup"
Sep 15 20:30:42.190886 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:42.190870    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-025acc06-lw8df"
Sep 15 20:30:42.193073 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:42.193054    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 20:30:42.193073 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:42.193065    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r5lpt\""
Sep 15 20:30:42.200265 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:42.200241    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-025acc06-lw8df"]
Sep 15 20:30:42.361248 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:42.361213    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05a3508f-b5d2-439f-b6b2-8a07fc422421-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-025acc06-lw8df\" (UID: \"05a3508f-b5d2-439f-b6b2-8a07fc422421\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-025acc06-lw8df"
Sep 15 20:30:42.462371 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:42.462341    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05a3508f-b5d2-439f-b6b2-8a07fc422421-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-025acc06-lw8df\" (UID: \"05a3508f-b5d2-439f-b6b2-8a07fc422421\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-025acc06-lw8df"
Sep 15 20:30:42.462941 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:42.462899    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05a3508f-b5d2-439f-b6b2-8a07fc422421-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-025acc06-lw8df\" (UID: \"05a3508f-b5d2-439f-b6b2-8a07fc422421\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-025acc06-lw8df"
Sep 15 20:30:42.502662 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:42.502640    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-025acc06-lw8df"
Sep 15 20:30:42.544017 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:42.543976    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:30:42.774270 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:42.774244    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-025acc06-lw8df"]
Sep 15 20:30:42.776261 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:30:42.776237    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05a3508f_b5d2_439f_b6b2_8a07fc422421.slice/crio-8a8ff674ad3ed7e9cd26f31e1b0d5717177499e4636bfccc246097cf6e197846 WatchSource:0}: Error finding container 8a8ff674ad3ed7e9cd26f31e1b0d5717177499e4636bfccc246097cf6e197846: Status 404 returned error can't find the container with id 8a8ff674ad3ed7e9cd26f31e1b0d5717177499e4636bfccc246097cf6e197846
Sep 15 20:30:43.173806 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:43.173725    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:30:43.236634 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:43.236603    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-025acc06-lw8df" event={"ID":"05a3508f-b5d2-439f-b6b2-8a07fc422421","Type":"ContainerStarted","Data":"9c0b569a594486293beff734330a826039445450b866c1afb1d8d0d39d196ae0"}
Sep 15 20:30:43.237105 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:43.236654    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-025acc06-lw8df" event={"ID":"05a3508f-b5d2-439f-b6b2-8a07fc422421","Type":"ContainerStarted","Data":"8a8ff674ad3ed7e9cd26f31e1b0d5717177499e4636bfccc246097cf6e197846"}
Sep 15 20:30:43.254322 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:43.254277    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-025acc06-lw8df" podStartSLOduration=1.254261158 podStartE2EDuration="1.254261158s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:30:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:30:43.252472072 +0000 UTC m=+1557.740936548" watchObservedRunningTime="2026-09-15 20:30:43.254261158 +0000 UTC m=+1557.742725665"
Sep 15 20:30:43.960420 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:43.960385    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:30:44.244077 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:44.243994    2576 generic.go:356] "Generic (PLEG): container finished" podID="05a3508f-b5d2-439f-b6b2-8a07fc422421" containerID="9c0b569a594486293beff734330a826039445450b866c1afb1d8d0d39d196ae0" exitCode=0
Sep 15 20:30:44.244077 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:44.244051    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-025acc06-lw8df" event={"ID":"05a3508f-b5d2-439f-b6b2-8a07fc422421","Type":"ContainerDied","Data":"9c0b569a594486293beff734330a826039445450b866c1afb1d8d0d39d196ae0"}
Sep 15 20:30:44.589582 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:44.589479    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:30:45.243621 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:45.243585    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:30:45.388315 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:45.388294    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-025acc06-lw8df"
Sep 15 20:30:45.391582 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:45.391557    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05a3508f-b5d2-439f-b6b2-8a07fc422421-maas-api-service-ca\") pod \"05a3508f-b5d2-439f-b6b2-8a07fc422421\" (UID: \"05a3508f-b5d2-439f-b6b2-8a07fc422421\") "
Sep 15 20:30:45.391856 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:45.391834    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05a3508f-b5d2-439f-b6b2-8a07fc422421-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "05a3508f-b5d2-439f-b6b2-8a07fc422421" (UID: "05a3508f-b5d2-439f-b6b2-8a07fc422421"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:30:45.493110 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:45.493077    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05a3508f-b5d2-439f-b6b2-8a07fc422421-maas-api-service-ca\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:30:45.925875 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:45.925842    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt"]
Sep 15 20:30:45.926328 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:45.926313    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05a3508f-b5d2-439f-b6b2-8a07fc422421" containerName="revoke-keys"
Sep 15 20:30:45.926328 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:45.926329    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="05a3508f-b5d2-439f-b6b2-8a07fc422421" containerName="revoke-keys"
Sep 15 20:30:45.926490 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:45.926411    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="05a3508f-b5d2-439f-b6b2-8a07fc422421" containerName="revoke-keys"
Sep 15 20:30:45.929660 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:45.929640    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt"
Sep 15 20:30:45.932050 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:45.932030    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-92702a\"/\"kube-root-ca.crt\""
Sep 15 20:30:45.932759 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:45.932739    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-92702a\"/\"auth-test-model-92702a-kserve-self-signed-certs\""
Sep 15 20:30:45.932856 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:45.932803    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-92702a\"/\"openshift-service-ca.crt\""
Sep 15 20:30:45.932856 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:45.932803    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-92702a\"/\"default-dockercfg-fcw5j\""
Sep 15 20:30:45.936513 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:45.936490    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt"]
Sep 15 20:30:45.949007 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:45.948981    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:30:45.998263 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:45.998237    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8aa6c347-9591-490f-bc74-4ee372e2db2f-model-cache\") pod \"auth-test-model-92702a-kserve-8c4d945ff-lm2dt\" (UID: \"8aa6c347-9591-490f-bc74-4ee372e2db2f\") " pod="ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt"
Sep 15 20:30:45.998366 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:45.998268    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8aa6c347-9591-490f-bc74-4ee372e2db2f-dshm\") pod \"auth-test-model-92702a-kserve-8c4d945ff-lm2dt\" (UID: \"8aa6c347-9591-490f-bc74-4ee372e2db2f\") " pod="ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt"
Sep 15 20:30:45.998366 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:45.998319    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8aa6c347-9591-490f-bc74-4ee372e2db2f-home\") pod \"auth-test-model-92702a-kserve-8c4d945ff-lm2dt\" (UID: \"8aa6c347-9591-490f-bc74-4ee372e2db2f\") " pod="ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt"
Sep 15 20:30:45.998366 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:45.998339    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8aa6c347-9591-490f-bc74-4ee372e2db2f-tmp-dir\") pod \"auth-test-model-92702a-kserve-8c4d945ff-lm2dt\" (UID: \"8aa6c347-9591-490f-bc74-4ee372e2db2f\") " pod="ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt"
Sep 15 20:30:45.998366 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:45.998359    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8aa6c347-9591-490f-bc74-4ee372e2db2f-tls-certs\") pod \"auth-test-model-92702a-kserve-8c4d945ff-lm2dt\" (UID: \"8aa6c347-9591-490f-bc74-4ee372e2db2f\") " pod="ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt"
Sep 15 20:30:45.998500 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:45.998437    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcllj\" (UniqueName: \"kubernetes.io/projected/8aa6c347-9591-490f-bc74-4ee372e2db2f-kube-api-access-pcllj\") pod \"auth-test-model-92702a-kserve-8c4d945ff-lm2dt\" (UID: \"8aa6c347-9591-490f-bc74-4ee372e2db2f\") " pod="ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt"
Sep 15 20:30:46.075143 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:46.075113    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:30:46.099750 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:46.099713    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8aa6c347-9591-490f-bc74-4ee372e2db2f-home\") pod \"auth-test-model-92702a-kserve-8c4d945ff-lm2dt\" (UID: \"8aa6c347-9591-490f-bc74-4ee372e2db2f\") " pod="ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt"
Sep 15 20:30:46.099886 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:46.099773    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8aa6c347-9591-490f-bc74-4ee372e2db2f-tmp-dir\") pod \"auth-test-model-92702a-kserve-8c4d945ff-lm2dt\" (UID: \"8aa6c347-9591-490f-bc74-4ee372e2db2f\") " pod="ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt"
Sep 15 20:30:46.099886 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:46.099809    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8aa6c347-9591-490f-bc74-4ee372e2db2f-tls-certs\") pod \"auth-test-model-92702a-kserve-8c4d945ff-lm2dt\" (UID: \"8aa6c347-9591-490f-bc74-4ee372e2db2f\") " pod="ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt"
Sep 15 20:30:46.099886 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:46.099876    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcllj\" (UniqueName: \"kubernetes.io/projected/8aa6c347-9591-490f-bc74-4ee372e2db2f-kube-api-access-pcllj\") pod \"auth-test-model-92702a-kserve-8c4d945ff-lm2dt\" (UID: \"8aa6c347-9591-490f-bc74-4ee372e2db2f\") " pod="ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt"
Sep 15 20:30:46.100078 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:46.099968    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8aa6c347-9591-490f-bc74-4ee372e2db2f-model-cache\") pod \"auth-test-model-92702a-kserve-8c4d945ff-lm2dt\" (UID: \"8aa6c347-9591-490f-bc74-4ee372e2db2f\") " pod="ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt"
Sep 15 20:30:46.100078 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:46.100002    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8aa6c347-9591-490f-bc74-4ee372e2db2f-dshm\") pod \"auth-test-model-92702a-kserve-8c4d945ff-lm2dt\" (UID: \"8aa6c347-9591-490f-bc74-4ee372e2db2f\") " pod="ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt"
Sep 15 20:30:46.100216 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:46.100185    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8aa6c347-9591-490f-bc74-4ee372e2db2f-home\") pod \"auth-test-model-92702a-kserve-8c4d945ff-lm2dt\" (UID: \"8aa6c347-9591-490f-bc74-4ee372e2db2f\") " pod="ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt"
Sep 15 20:30:46.100341 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:46.100230    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8aa6c347-9591-490f-bc74-4ee372e2db2f-tmp-dir\") pod \"auth-test-model-92702a-kserve-8c4d945ff-lm2dt\" (UID: \"8aa6c347-9591-490f-bc74-4ee372e2db2f\") " pod="ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt"
Sep 15 20:30:46.100408 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:46.100344    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8aa6c347-9591-490f-bc74-4ee372e2db2f-model-cache\") pod \"auth-test-model-92702a-kserve-8c4d945ff-lm2dt\" (UID: \"8aa6c347-9591-490f-bc74-4ee372e2db2f\") " pod="ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt"
Sep 15 20:30:46.102104 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:46.102082    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8aa6c347-9591-490f-bc74-4ee372e2db2f-dshm\") pod \"auth-test-model-92702a-kserve-8c4d945ff-lm2dt\" (UID: \"8aa6c347-9591-490f-bc74-4ee372e2db2f\") " pod="ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt"
Sep 15 20:30:46.102460 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:46.102439    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8aa6c347-9591-490f-bc74-4ee372e2db2f-tls-certs\") pod \"auth-test-model-92702a-kserve-8c4d945ff-lm2dt\" (UID: \"8aa6c347-9591-490f-bc74-4ee372e2db2f\") " pod="ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt"
Sep 15 20:30:46.107036 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:46.107015    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcllj\" (UniqueName: \"kubernetes.io/projected/8aa6c347-9591-490f-bc74-4ee372e2db2f-kube-api-access-pcllj\") pod \"auth-test-model-92702a-kserve-8c4d945ff-lm2dt\" (UID: \"8aa6c347-9591-490f-bc74-4ee372e2db2f\") " pod="ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt"
Sep 15 20:30:46.241715 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:46.241693    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt"
Sep 15 20:30:46.255224 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:46.255198    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-025acc06-lw8df" event={"ID":"05a3508f-b5d2-439f-b6b2-8a07fc422421","Type":"ContainerDied","Data":"8a8ff674ad3ed7e9cd26f31e1b0d5717177499e4636bfccc246097cf6e197846"}
Sep 15 20:30:46.255333 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:46.255219    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-025acc06-lw8df"
Sep 15 20:30:46.255333 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:46.255239    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8a8ff674ad3ed7e9cd26f31e1b0d5717177499e4636bfccc246097cf6e197846"
Sep 15 20:30:46.514342 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:46.514316    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt"]
Sep 15 20:30:46.515753 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:30:46.515728    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8aa6c347_9591_490f_bc74_4ee372e2db2f.slice/crio-82af52a420733221f5ddf790c64e9b14af184477a3e1a1b5abe693a6a2e5b166 WatchSource:0}: Error finding container 82af52a420733221f5ddf790c64e9b14af184477a3e1a1b5abe693a6a2e5b166: Status 404 returned error can't find the container with id 82af52a420733221f5ddf790c64e9b14af184477a3e1a1b5abe693a6a2e5b166
Sep 15 20:30:46.728585 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:46.728550    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:30:47.262849 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:47.262814    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt" event={"ID":"8aa6c347-9591-490f-bc74-4ee372e2db2f","Type":"ContainerStarted","Data":"d06c91f6eca55744eca755a3abc216b1a6f87c920031bb2b304186bef9434036"}
Sep 15 20:30:47.262849 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:47.262854    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt" event={"ID":"8aa6c347-9591-490f-bc74-4ee372e2db2f","Type":"ContainerStarted","Data":"82af52a420733221f5ddf790c64e9b14af184477a3e1a1b5abe693a6a2e5b166"}
Sep 15 20:30:47.279960 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:47.279892    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt" podStartSLOduration=2.27987717 podStartE2EDuration="2.27987717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:30:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:30:47.277975134 +0000 UTC m=+1561.766439603" watchObservedRunningTime="2026-09-15 20:30:47.27987717 +0000 UTC m=+1561.768341645"
Sep 15 20:30:47.592069 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:47.591982    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:30:48.263434 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:48.263395    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:30:48.932372 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:48.932341    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:30:56.242739 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:56.242678    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt"
Sep 15 20:30:56.243257 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:56.242752    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt"
Sep 15 20:30:56.255204 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:56.255183    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt"
Sep 15 20:30:56.323340 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:56.323312    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt"
Sep 15 20:30:57.675070 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:57.675039    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:30:57.716000 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:57.715965    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-025acc06-8c68f8b65-frsqn"]
Sep 15 20:30:57.716259 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:57.716208    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-025acc06-8c68f8b65-frsqn" podUID="f2e624f3-e195-41d8-ad2d-36490fdd0edd" containerName="payload-processing" containerID="cri-o://aa87c4817ebd8b9e5fdd2a18d79b2ace721101caa71a71384f7fad83ca7e6c98" gracePeriod=30
Sep 15 20:30:57.722843 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:57.722715    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-025acc06-85c48fc6bb-plv6p"]
Sep 15 20:30:57.722995 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:57.722956    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-025acc06-85c48fc6bb-plv6p" podUID="05f58653-cd09-44c9-a92a-e282a5c78e83" containerName="payload-pre-processing" containerID="cri-o://0e85809f6c6dfd6b94ef7dd40cc3f999052a90f4778f73398f9097c2ccf6647d" gracePeriod=30
Sep 15 20:30:57.747496 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:57.747459    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:30:57.848251 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:57.848211    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:30:57.881982 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:57.881950    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-025acc06-lw8df"]
Sep 15 20:30:57.885370 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:57.885347    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-025acc06-lw8df"]
Sep 15 20:30:57.951314 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:57.951270    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:30:57.979237 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:57.979214    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-025acc06-8c68f8b65-frsqn"
Sep 15 20:30:58.007302 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:58.007275    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-025acc06-85c48fc6bb-plv6p"
Sep 15 20:30:58.115849 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:58.115807    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2e624f3-e195-41d8-ad2d-36490fdd0edd-plugins-config-volume\") pod \"f2e624f3-e195-41d8-ad2d-36490fdd0edd\" (UID: \"f2e624f3-e195-41d8-ad2d-36490fdd0edd\") "
Sep 15 20:30:58.115849 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:58.115855    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7rdw\" (UniqueName: \"kubernetes.io/projected/f2e624f3-e195-41d8-ad2d-36490fdd0edd-kube-api-access-b7rdw\") pod \"f2e624f3-e195-41d8-ad2d-36490fdd0edd\" (UID: \"f2e624f3-e195-41d8-ad2d-36490fdd0edd\") "
Sep 15 20:30:58.116128 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:58.115903    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05f58653-cd09-44c9-a92a-e282a5c78e83-plugins-config-volume\") pod \"05f58653-cd09-44c9-a92a-e282a5c78e83\" (UID: \"05f58653-cd09-44c9-a92a-e282a5c78e83\") "
Sep 15 20:30:58.116128 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:58.115979    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4769h\" (UniqueName: \"kubernetes.io/projected/05f58653-cd09-44c9-a92a-e282a5c78e83-kube-api-access-4769h\") pod \"05f58653-cd09-44c9-a92a-e282a5c78e83\" (UID: \"05f58653-cd09-44c9-a92a-e282a5c78e83\") "
Sep 15 20:30:58.116286 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:58.116264    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2e624f3-e195-41d8-ad2d-36490fdd0edd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f2e624f3-e195-41d8-ad2d-36490fdd0edd" (UID: "f2e624f3-e195-41d8-ad2d-36490fdd0edd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:30:58.116338 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:58.116301    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05f58653-cd09-44c9-a92a-e282a5c78e83-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "05f58653-cd09-44c9-a92a-e282a5c78e83" (UID: "05f58653-cd09-44c9-a92a-e282a5c78e83"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:30:58.117943 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:58.117910    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2e624f3-e195-41d8-ad2d-36490fdd0edd-kube-api-access-b7rdw" (OuterVolumeSpecName: "kube-api-access-b7rdw") pod "f2e624f3-e195-41d8-ad2d-36490fdd0edd" (UID: "f2e624f3-e195-41d8-ad2d-36490fdd0edd"). InnerVolumeSpecName "kube-api-access-b7rdw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:30:58.118051 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:58.118034    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05f58653-cd09-44c9-a92a-e282a5c78e83-kube-api-access-4769h" (OuterVolumeSpecName: "kube-api-access-4769h") pod "05f58653-cd09-44c9-a92a-e282a5c78e83" (UID: "05f58653-cd09-44c9-a92a-e282a5c78e83"). InnerVolumeSpecName "kube-api-access-4769h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:30:58.143753 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:58.143723    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05a3508f-b5d2-439f-b6b2-8a07fc422421" path="/var/lib/kubelet/pods/05a3508f-b5d2-439f-b6b2-8a07fc422421/volumes"
Sep 15 20:30:58.189553 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:58.189461    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:30:58.217747 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:58.217724    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2e624f3-e195-41d8-ad2d-36490fdd0edd-plugins-config-volume\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:30:58.217747 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:58.217744    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7rdw\" (UniqueName: \"kubernetes.io/projected/f2e624f3-e195-41d8-ad2d-36490fdd0edd-kube-api-access-b7rdw\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:30:58.217888 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:58.217756    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05f58653-cd09-44c9-a92a-e282a5c78e83-plugins-config-volume\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:30:58.217888 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:58.217766    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4769h\" (UniqueName: \"kubernetes.io/projected/05f58653-cd09-44c9-a92a-e282a5c78e83-kube-api-access-4769h\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:30:58.324713 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:58.324676    2576 generic.go:356] "Generic (PLEG): container finished" podID="05f58653-cd09-44c9-a92a-e282a5c78e83" containerID="0e85809f6c6dfd6b94ef7dd40cc3f999052a90f4778f73398f9097c2ccf6647d" exitCode=0
Sep 15 20:30:58.324868 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:58.324725    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-025acc06-85c48fc6bb-plv6p"
Sep 15 20:30:58.324868 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:58.324763    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-025acc06-85c48fc6bb-plv6p" event={"ID":"05f58653-cd09-44c9-a92a-e282a5c78e83","Type":"ContainerDied","Data":"0e85809f6c6dfd6b94ef7dd40cc3f999052a90f4778f73398f9097c2ccf6647d"}
Sep 15 20:30:58.324868 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:58.324823    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-025acc06-85c48fc6bb-plv6p" event={"ID":"05f58653-cd09-44c9-a92a-e282a5c78e83","Type":"ContainerDied","Data":"42356f65a844ec09823a9b075a3a0ff2b9cab2284f175356ea0dedf250ef9122"}
Sep 15 20:30:58.324868 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:58.324857    2576 scope.go:117] "RemoveContainer" containerID="0e85809f6c6dfd6b94ef7dd40cc3f999052a90f4778f73398f9097c2ccf6647d"
Sep 15 20:30:58.326314 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:58.326291    2576 generic.go:356] "Generic (PLEG): container finished" podID="f2e624f3-e195-41d8-ad2d-36490fdd0edd" containerID="aa87c4817ebd8b9e5fdd2a18d79b2ace721101caa71a71384f7fad83ca7e6c98" exitCode=0
Sep 15 20:30:58.326405 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:58.326339    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-025acc06-8c68f8b65-frsqn"
Sep 15 20:30:58.326405 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:58.326343    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-025acc06-8c68f8b65-frsqn" event={"ID":"f2e624f3-e195-41d8-ad2d-36490fdd0edd","Type":"ContainerDied","Data":"aa87c4817ebd8b9e5fdd2a18d79b2ace721101caa71a71384f7fad83ca7e6c98"}
Sep 15 20:30:58.326405 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:58.326394    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-025acc06-8c68f8b65-frsqn" event={"ID":"f2e624f3-e195-41d8-ad2d-36490fdd0edd","Type":"ContainerDied","Data":"fc23699c853a5a37214ed8ca234c6df0a1cc5e0b36c16520bb49c9515f9d19e7"}
Sep 15 20:30:58.334126 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:58.334101    2576 scope.go:117] "RemoveContainer" containerID="0e85809f6c6dfd6b94ef7dd40cc3f999052a90f4778f73398f9097c2ccf6647d"
Sep 15 20:30:58.334395 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:30:58.334376    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e85809f6c6dfd6b94ef7dd40cc3f999052a90f4778f73398f9097c2ccf6647d\": container with ID starting with 0e85809f6c6dfd6b94ef7dd40cc3f999052a90f4778f73398f9097c2ccf6647d not found: ID does not exist" containerID="0e85809f6c6dfd6b94ef7dd40cc3f999052a90f4778f73398f9097c2ccf6647d"
Sep 15 20:30:58.334488 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:58.334403    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e85809f6c6dfd6b94ef7dd40cc3f999052a90f4778f73398f9097c2ccf6647d"} err="failed to get container status \"0e85809f6c6dfd6b94ef7dd40cc3f999052a90f4778f73398f9097c2ccf6647d\": rpc error: code = NotFound desc = could not find container \"0e85809f6c6dfd6b94ef7dd40cc3f999052a90f4778f73398f9097c2ccf6647d\": container with ID starting with 0e85809f6c6dfd6b94ef7dd40cc3f999052a90f4778f73398f9097c2ccf6647d not found: ID does not exist"
Sep 15 20:30:58.334488 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:58.334418    2576 scope.go:117] "RemoveContainer" containerID="aa87c4817ebd8b9e5fdd2a18d79b2ace721101caa71a71384f7fad83ca7e6c98"
Sep 15 20:30:58.342276 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:58.342259    2576 scope.go:117] "RemoveContainer" containerID="aa87c4817ebd8b9e5fdd2a18d79b2ace721101caa71a71384f7fad83ca7e6c98"
Sep 15 20:30:58.342546 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:30:58.342529    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa87c4817ebd8b9e5fdd2a18d79b2ace721101caa71a71384f7fad83ca7e6c98\": container with ID starting with aa87c4817ebd8b9e5fdd2a18d79b2ace721101caa71a71384f7fad83ca7e6c98 not found: ID does not exist" containerID="aa87c4817ebd8b9e5fdd2a18d79b2ace721101caa71a71384f7fad83ca7e6c98"
Sep 15 20:30:58.342609 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:58.342551    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa87c4817ebd8b9e5fdd2a18d79b2ace721101caa71a71384f7fad83ca7e6c98"} err="failed to get container status \"aa87c4817ebd8b9e5fdd2a18d79b2ace721101caa71a71384f7fad83ca7e6c98\": rpc error: code = NotFound desc = could not find container \"aa87c4817ebd8b9e5fdd2a18d79b2ace721101caa71a71384f7fad83ca7e6c98\": container with ID starting with aa87c4817ebd8b9e5fdd2a18d79b2ace721101caa71a71384f7fad83ca7e6c98 not found: ID does not exist"
Sep 15 20:30:58.348473 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:58.348453    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-025acc06-8c68f8b65-frsqn"]
Sep 15 20:30:58.359307 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:58.359280    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-025acc06-8c68f8b65-frsqn"]
Sep 15 20:30:58.365432 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:58.365413    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-025acc06-85c48fc6bb-plv6p"]
Sep 15 20:30:58.373052 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:58.373032    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-025acc06-85c48fc6bb-plv6p"]
Sep 15 20:30:58.959837 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:58.959798    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:30:59.633168 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:30:59.633136    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:31:00.143334 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:00.143292    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05f58653-cd09-44c9-a92a-e282a5c78e83" path="/var/lib/kubelet/pods/05f58653-cd09-44c9-a92a-e282a5c78e83/volumes"
Sep 15 20:31:00.143875 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:00.143852    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2e624f3-e195-41d8-ad2d-36490fdd0edd" path="/var/lib/kubelet/pods/f2e624f3-e195-41d8-ad2d-36490fdd0edd/volumes"
Sep 15 20:31:00.269087 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:00.269051    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:31:00.931547 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:00.931511    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:31:01.057770 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:01.057733    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:31:01.584062 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:01.584030    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:31:01.859221 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:01.859147    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:31:02.532607 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:02.532573    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:31:03.104559 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:03.104522    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:31:06.560595 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:06.560558    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h"]
Sep 15 20:31:06.603543 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:06.603516    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:31:09.433635 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:09.433600    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf"]
Sep 15 20:31:09.434047 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:09.434033    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2e624f3-e195-41d8-ad2d-36490fdd0edd" containerName="payload-processing"
Sep 15 20:31:09.434095 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:09.434049    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2e624f3-e195-41d8-ad2d-36490fdd0edd" containerName="payload-processing"
Sep 15 20:31:09.434095 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:09.434063    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05f58653-cd09-44c9-a92a-e282a5c78e83" containerName="payload-pre-processing"
Sep 15 20:31:09.434095 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:09.434069    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="05f58653-cd09-44c9-a92a-e282a5c78e83" containerName="payload-pre-processing"
Sep 15 20:31:09.434187 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:09.434148    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="05f58653-cd09-44c9-a92a-e282a5c78e83" containerName="payload-pre-processing"
Sep 15 20:31:09.434187 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:09.434155    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2e624f3-e195-41d8-ad2d-36490fdd0edd" containerName="payload-processing"
Sep 15 20:31:09.438708 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:09.438692    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf"
Sep 15 20:31:09.441225 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:09.441203    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-725e67\"/\"distinct-w4-725e67-kserve-self-signed-certs\""
Sep 15 20:31:09.447253 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:09.447231    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf"]
Sep 15 20:31:09.527621 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:09.527588    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-tmp-dir\") pod \"distinct-w4-725e67-kserve-84475bc795-hpscf\" (UID: \"b1af4267-cc2b-4a2a-8141-ef0a64aa29cf\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf"
Sep 15 20:31:09.527621 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:09.527626    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-model-cache\") pod \"distinct-w4-725e67-kserve-84475bc795-hpscf\" (UID: \"b1af4267-cc2b-4a2a-8141-ef0a64aa29cf\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf"
Sep 15 20:31:09.527835 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:09.527649    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-dshm\") pod \"distinct-w4-725e67-kserve-84475bc795-hpscf\" (UID: \"b1af4267-cc2b-4a2a-8141-ef0a64aa29cf\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf"
Sep 15 20:31:09.527835 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:09.527763    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-tls-certs\") pod \"distinct-w4-725e67-kserve-84475bc795-hpscf\" (UID: \"b1af4267-cc2b-4a2a-8141-ef0a64aa29cf\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf"
Sep 15 20:31:09.527835 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:09.527820    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vv69d\" (UniqueName: \"kubernetes.io/projected/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-kube-api-access-vv69d\") pod \"distinct-w4-725e67-kserve-84475bc795-hpscf\" (UID: \"b1af4267-cc2b-4a2a-8141-ef0a64aa29cf\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf"
Sep 15 20:31:09.527965 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:09.527856    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-home\") pod \"distinct-w4-725e67-kserve-84475bc795-hpscf\" (UID: \"b1af4267-cc2b-4a2a-8141-ef0a64aa29cf\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf"
Sep 15 20:31:09.533044 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:09.533017    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:31:09.629011 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:09.628981    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-tls-certs\") pod \"distinct-w4-725e67-kserve-84475bc795-hpscf\" (UID: \"b1af4267-cc2b-4a2a-8141-ef0a64aa29cf\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf"
Sep 15 20:31:09.629175 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:09.629038    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv69d\" (UniqueName: \"kubernetes.io/projected/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-kube-api-access-vv69d\") pod \"distinct-w4-725e67-kserve-84475bc795-hpscf\" (UID: \"b1af4267-cc2b-4a2a-8141-ef0a64aa29cf\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf"
Sep 15 20:31:09.629235 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:09.629210    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-home\") pod \"distinct-w4-725e67-kserve-84475bc795-hpscf\" (UID: \"b1af4267-cc2b-4a2a-8141-ef0a64aa29cf\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf"
Sep 15 20:31:09.629306 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:09.629289    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-tmp-dir\") pod \"distinct-w4-725e67-kserve-84475bc795-hpscf\" (UID: \"b1af4267-cc2b-4a2a-8141-ef0a64aa29cf\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf"
Sep 15 20:31:09.629367 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:09.629320    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-model-cache\") pod \"distinct-w4-725e67-kserve-84475bc795-hpscf\" (UID: \"b1af4267-cc2b-4a2a-8141-ef0a64aa29cf\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf"
Sep 15 20:31:09.629367 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:09.629347    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-dshm\") pod \"distinct-w4-725e67-kserve-84475bc795-hpscf\" (UID: \"b1af4267-cc2b-4a2a-8141-ef0a64aa29cf\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf"
Sep 15 20:31:09.629587 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:09.629568    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-home\") pod \"distinct-w4-725e67-kserve-84475bc795-hpscf\" (UID: \"b1af4267-cc2b-4a2a-8141-ef0a64aa29cf\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf"
Sep 15 20:31:09.629656 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:09.629637    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-tmp-dir\") pod \"distinct-w4-725e67-kserve-84475bc795-hpscf\" (UID: \"b1af4267-cc2b-4a2a-8141-ef0a64aa29cf\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf"
Sep 15 20:31:09.629704 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:09.629664    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-model-cache\") pod \"distinct-w4-725e67-kserve-84475bc795-hpscf\" (UID: \"b1af4267-cc2b-4a2a-8141-ef0a64aa29cf\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf"
Sep 15 20:31:09.631446 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:09.631426    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-tls-certs\") pod \"distinct-w4-725e67-kserve-84475bc795-hpscf\" (UID: \"b1af4267-cc2b-4a2a-8141-ef0a64aa29cf\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf"
Sep 15 20:31:09.631556 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:09.631483    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-dshm\") pod \"distinct-w4-725e67-kserve-84475bc795-hpscf\" (UID: \"b1af4267-cc2b-4a2a-8141-ef0a64aa29cf\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf"
Sep 15 20:31:09.636678 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:09.636632    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv69d\" (UniqueName: \"kubernetes.io/projected/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-kube-api-access-vv69d\") pod \"distinct-w4-725e67-kserve-84475bc795-hpscf\" (UID: \"b1af4267-cc2b-4a2a-8141-ef0a64aa29cf\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf"
Sep 15 20:31:09.749972 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:09.749949    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf"
Sep 15 20:31:10.019581 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:10.019559    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf"]
Sep 15 20:31:10.020846 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:31:10.020814    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb1af4267_cc2b_4a2a_8141_ef0a64aa29cf.slice/crio-62cf0b0a89d6d2ddeb753d036eb483c08deda861d9f286610db43349de47d348 WatchSource:0}: Error finding container 62cf0b0a89d6d2ddeb753d036eb483c08deda861d9f286610db43349de47d348: Status 404 returned error can't find the container with id 62cf0b0a89d6d2ddeb753d036eb483c08deda861d9f286610db43349de47d348
Sep 15 20:31:10.395202 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:10.395105    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf" event={"ID":"b1af4267-cc2b-4a2a-8141-ef0a64aa29cf","Type":"ContainerStarted","Data":"2121178f8b3af9ded180cd6c23ba21f74aea07af881727a55b1cd9d076efbb3a"}
Sep 15 20:31:10.395202 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:10.395155    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf" event={"ID":"b1af4267-cc2b-4a2a-8141-ef0a64aa29cf","Type":"ContainerStarted","Data":"62cf0b0a89d6d2ddeb753d036eb483c08deda861d9f286610db43349de47d348"}
Sep 15 20:31:10.413828 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:10.413781    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf" podStartSLOduration=1.413767171 podStartE2EDuration="1.413767171s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:31:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:31:10.412012334 +0000 UTC m=+1584.900476812" watchObservedRunningTime="2026-09-15 20:31:10.413767171 +0000 UTC m=+1584.902231646"
Sep 15 20:31:16.529196 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:16.529162    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:31:19.750356 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:19.750320    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf"
Sep 15 20:31:19.750723 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:19.750366    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf"
Sep 15 20:31:19.762723 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:19.762702    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf"
Sep 15 20:31:20.459255 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:20.459224    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf"
Sep 15 20:31:21.579931 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:21.579891    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h" podUID="822b13dc-6e4a-468e-92ff-70fc7b2a55aa" containerName="main" containerID="cri-o://b972e0770a65151a6bddc1d3feab017b461e3f5934b310591352937df5ed1d6d" gracePeriod=45
Sep 15 20:31:21.838688 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:21.838625    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h"
Sep 15 20:31:21.842364 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:21.842340    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-dshm\") pod \"822b13dc-6e4a-468e-92ff-70fc7b2a55aa\" (UID: \"822b13dc-6e4a-468e-92ff-70fc7b2a55aa\") "
Sep 15 20:31:21.842467 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:21.842406    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5j8tx\" (UniqueName: \"kubernetes.io/projected/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-kube-api-access-5j8tx\") pod \"822b13dc-6e4a-468e-92ff-70fc7b2a55aa\" (UID: \"822b13dc-6e4a-468e-92ff-70fc7b2a55aa\") "
Sep 15 20:31:21.842520 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:21.842478    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-tmp-dir\") pod \"822b13dc-6e4a-468e-92ff-70fc7b2a55aa\" (UID: \"822b13dc-6e4a-468e-92ff-70fc7b2a55aa\") "
Sep 15 20:31:21.842562 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:21.842552    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-model-cache\") pod \"822b13dc-6e4a-468e-92ff-70fc7b2a55aa\" (UID: \"822b13dc-6e4a-468e-92ff-70fc7b2a55aa\") "
Sep 15 20:31:21.842608 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:21.842594    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-tls-certs\") pod \"822b13dc-6e4a-468e-92ff-70fc7b2a55aa\" (UID: \"822b13dc-6e4a-468e-92ff-70fc7b2a55aa\") "
Sep 15 20:31:21.842654 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:21.842618    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-home\") pod \"822b13dc-6e4a-468e-92ff-70fc7b2a55aa\" (UID: \"822b13dc-6e4a-468e-92ff-70fc7b2a55aa\") "
Sep 15 20:31:21.842743 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:21.842718    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "822b13dc-6e4a-468e-92ff-70fc7b2a55aa" (UID: "822b13dc-6e4a-468e-92ff-70fc7b2a55aa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:31:21.842830 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:21.842813    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-model-cache" (OuterVolumeSpecName: "model-cache") pod "822b13dc-6e4a-468e-92ff-70fc7b2a55aa" (UID: "822b13dc-6e4a-468e-92ff-70fc7b2a55aa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:31:21.842953 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:21.842894    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-home" (OuterVolumeSpecName: "home") pod "822b13dc-6e4a-468e-92ff-70fc7b2a55aa" (UID: "822b13dc-6e4a-468e-92ff-70fc7b2a55aa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:31:21.843059 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:21.843041    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-model-cache\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:31:21.843139 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:21.843064    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-home\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:31:21.843139 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:21.843076    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-tmp-dir\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:31:21.844905 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:21.844884    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "822b13dc-6e4a-468e-92ff-70fc7b2a55aa" (UID: "822b13dc-6e4a-468e-92ff-70fc7b2a55aa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:31:21.845018 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:21.845001    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-dshm" (OuterVolumeSpecName: "dshm") pod "822b13dc-6e4a-468e-92ff-70fc7b2a55aa" (UID: "822b13dc-6e4a-468e-92ff-70fc7b2a55aa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:31:21.845071 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:21.845024    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-kube-api-access-5j8tx" (OuterVolumeSpecName: "kube-api-access-5j8tx") pod "822b13dc-6e4a-468e-92ff-70fc7b2a55aa" (UID: "822b13dc-6e4a-468e-92ff-70fc7b2a55aa"). InnerVolumeSpecName "kube-api-access-5j8tx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:31:21.943605 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:21.943573    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5j8tx\" (UniqueName: \"kubernetes.io/projected/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-kube-api-access-5j8tx\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:31:21.943605 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:21.943599    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-tls-certs\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:31:21.943605 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:21.943610    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/822b13dc-6e4a-468e-92ff-70fc7b2a55aa-dshm\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:31:22.459811 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:22.459783    2576 generic.go:356] "Generic (PLEG): container finished" podID="822b13dc-6e4a-468e-92ff-70fc7b2a55aa" containerID="b972e0770a65151a6bddc1d3feab017b461e3f5934b310591352937df5ed1d6d" exitCode=0
Sep 15 20:31:22.459986 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:22.459823    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h"
Sep 15 20:31:22.459986 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:22.459863    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h" event={"ID":"822b13dc-6e4a-468e-92ff-70fc7b2a55aa","Type":"ContainerDied","Data":"b972e0770a65151a6bddc1d3feab017b461e3f5934b310591352937df5ed1d6d"}
Sep 15 20:31:22.459986 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:22.459907    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h" event={"ID":"822b13dc-6e4a-468e-92ff-70fc7b2a55aa","Type":"ContainerDied","Data":"b7662f21d0164ac76866bee5e0f23f548c5b99688a0e3ea2becdc356b9d7328d"}
Sep 15 20:31:22.459986 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:22.459946    2576 scope.go:117] "RemoveContainer" containerID="b972e0770a65151a6bddc1d3feab017b461e3f5934b310591352937df5ed1d6d"
Sep 15 20:31:22.472183 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:22.472153    2576 scope.go:117] "RemoveContainer" containerID="b972e0770a65151a6bddc1d3feab017b461e3f5934b310591352937df5ed1d6d"
Sep 15 20:31:22.472595 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:31:22.472575    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b972e0770a65151a6bddc1d3feab017b461e3f5934b310591352937df5ed1d6d\": container with ID starting with b972e0770a65151a6bddc1d3feab017b461e3f5934b310591352937df5ed1d6d not found: ID does not exist" containerID="b972e0770a65151a6bddc1d3feab017b461e3f5934b310591352937df5ed1d6d"
Sep 15 20:31:22.472665 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:22.472605    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b972e0770a65151a6bddc1d3feab017b461e3f5934b310591352937df5ed1d6d"} err="failed to get container status \"b972e0770a65151a6bddc1d3feab017b461e3f5934b310591352937df5ed1d6d\": rpc error: code = NotFound desc = could not find container \"b972e0770a65151a6bddc1d3feab017b461e3f5934b310591352937df5ed1d6d\": container with ID starting with b972e0770a65151a6bddc1d3feab017b461e3f5934b310591352937df5ed1d6d not found: ID does not exist"
Sep 15 20:31:22.476112 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:22.476090    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h"]
Sep 15 20:31:22.479061 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:22.479039    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-025acc06/e2e-lifecycle-model-025acc06-kserve-5ff89f49d7-q789h"]
Sep 15 20:31:24.144041 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:24.144006    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="822b13dc-6e4a-468e-92ff-70fc7b2a55aa" path="/var/lib/kubelet/pods/822b13dc-6e4a-468e-92ff-70fc7b2a55aa/volumes"
Sep 15 20:31:43.391324 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:43.391213    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:31:43.538029 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:43.537981    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:31:49.130895 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:49.130846    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:31:49.231218 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:49.231184    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:31:49.442004 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:49.441896    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:31:50.775111 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:50.775077    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:31:50.875019 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:50.874986    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:31:50.975409 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:50.975364    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:31:51.692495 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:51.692465    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:31:51.786622 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:51.786589    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:31:51.890590 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:51.890561    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:31:53.389357 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:53.389325    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:31:55.093995 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:55.093948    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:31:59.311797 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:59.311762    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:31:59.374984 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:31:59.374957    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:32:00.054472 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:00.054438    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:32:05.757328 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:05.757292    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:32:05.861241 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:05.861208    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:32:05.960326 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:05.960292    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:32:08.618111 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:08.618062    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr"]
Sep 15 20:32:08.618734 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:08.618713    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="822b13dc-6e4a-468e-92ff-70fc7b2a55aa" containerName="main"
Sep 15 20:32:08.618734 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:08.618734    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="822b13dc-6e4a-468e-92ff-70fc7b2a55aa" containerName="main"
Sep 15 20:32:08.618897 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:08.618854    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="822b13dc-6e4a-468e-92ff-70fc7b2a55aa" containerName="main"
Sep 15 20:32:08.622383 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:08.622363    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr"
Sep 15 20:32:08.624600 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:08.624570    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-725e67\"/\"distinct-2-w4-725e67-kserve-self-signed-certs\""
Sep 15 20:32:08.629940 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:08.629904    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr"]
Sep 15 20:32:08.653634 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:08.653603    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:32:08.688148 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:08.688120    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xqzm\" (UniqueName: \"kubernetes.io/projected/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-kube-api-access-2xqzm\") pod \"distinct-2-w4-725e67-kserve-c988ccb4-6s6gr\" (UID: \"2b77ce5f-d129-4eb5-966c-aae7ad0c1d04\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr"
Sep 15 20:32:08.688278 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:08.688179    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-tls-certs\") pod \"distinct-2-w4-725e67-kserve-c988ccb4-6s6gr\" (UID: \"2b77ce5f-d129-4eb5-966c-aae7ad0c1d04\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr"
Sep 15 20:32:08.688278 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:08.688241    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-home\") pod \"distinct-2-w4-725e67-kserve-c988ccb4-6s6gr\" (UID: \"2b77ce5f-d129-4eb5-966c-aae7ad0c1d04\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr"
Sep 15 20:32:08.688369 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:08.688287    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-model-cache\") pod \"distinct-2-w4-725e67-kserve-c988ccb4-6s6gr\" (UID: \"2b77ce5f-d129-4eb5-966c-aae7ad0c1d04\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr"
Sep 15 20:32:08.688369 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:08.688306    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-tmp-dir\") pod \"distinct-2-w4-725e67-kserve-c988ccb4-6s6gr\" (UID: \"2b77ce5f-d129-4eb5-966c-aae7ad0c1d04\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr"
Sep 15 20:32:08.688443 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:08.688407    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-dshm\") pod \"distinct-2-w4-725e67-kserve-c988ccb4-6s6gr\" (UID: \"2b77ce5f-d129-4eb5-966c-aae7ad0c1d04\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr"
Sep 15 20:32:08.789381 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:08.789357    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-dshm\") pod \"distinct-2-w4-725e67-kserve-c988ccb4-6s6gr\" (UID: \"2b77ce5f-d129-4eb5-966c-aae7ad0c1d04\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr"
Sep 15 20:32:08.789537 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:08.789397    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xqzm\" (UniqueName: \"kubernetes.io/projected/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-kube-api-access-2xqzm\") pod \"distinct-2-w4-725e67-kserve-c988ccb4-6s6gr\" (UID: \"2b77ce5f-d129-4eb5-966c-aae7ad0c1d04\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr"
Sep 15 20:32:08.789537 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:08.789444    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-tls-certs\") pod \"distinct-2-w4-725e67-kserve-c988ccb4-6s6gr\" (UID: \"2b77ce5f-d129-4eb5-966c-aae7ad0c1d04\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr"
Sep 15 20:32:08.789537 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:08.789473    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-home\") pod \"distinct-2-w4-725e67-kserve-c988ccb4-6s6gr\" (UID: \"2b77ce5f-d129-4eb5-966c-aae7ad0c1d04\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr"
Sep 15 20:32:08.789537 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:08.789507    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-model-cache\") pod \"distinct-2-w4-725e67-kserve-c988ccb4-6s6gr\" (UID: \"2b77ce5f-d129-4eb5-966c-aae7ad0c1d04\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr"
Sep 15 20:32:08.789537 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:08.789534    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-tmp-dir\") pod \"distinct-2-w4-725e67-kserve-c988ccb4-6s6gr\" (UID: \"2b77ce5f-d129-4eb5-966c-aae7ad0c1d04\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr"
Sep 15 20:32:08.789912 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:08.789889    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-home\") pod \"distinct-2-w4-725e67-kserve-c988ccb4-6s6gr\" (UID: \"2b77ce5f-d129-4eb5-966c-aae7ad0c1d04\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr"
Sep 15 20:32:08.790019 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:08.789967    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-model-cache\") pod \"distinct-2-w4-725e67-kserve-c988ccb4-6s6gr\" (UID: \"2b77ce5f-d129-4eb5-966c-aae7ad0c1d04\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr"
Sep 15 20:32:08.790019 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:08.789981    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-tmp-dir\") pod \"distinct-2-w4-725e67-kserve-c988ccb4-6s6gr\" (UID: \"2b77ce5f-d129-4eb5-966c-aae7ad0c1d04\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr"
Sep 15 20:32:08.791683 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:08.791666    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-dshm\") pod \"distinct-2-w4-725e67-kserve-c988ccb4-6s6gr\" (UID: \"2b77ce5f-d129-4eb5-966c-aae7ad0c1d04\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr"
Sep 15 20:32:08.791864 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:08.791848    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-tls-certs\") pod \"distinct-2-w4-725e67-kserve-c988ccb4-6s6gr\" (UID: \"2b77ce5f-d129-4eb5-966c-aae7ad0c1d04\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr"
Sep 15 20:32:08.796842 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:08.796825    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xqzm\" (UniqueName: \"kubernetes.io/projected/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-kube-api-access-2xqzm\") pod \"distinct-2-w4-725e67-kserve-c988ccb4-6s6gr\" (UID: \"2b77ce5f-d129-4eb5-966c-aae7ad0c1d04\") " pod="e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr"
Sep 15 20:32:08.933875 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:08.933784    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr"
Sep 15 20:32:09.203274 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:09.203243    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr"]
Sep 15 20:32:09.204839 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:32:09.204813    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b77ce5f_d129_4eb5_966c_aae7ad0c1d04.slice/crio-a09fef78107215983939a82a37e87d93b6f2b7d3432254565c75c1ef73e54f90 WatchSource:0}: Error finding container a09fef78107215983939a82a37e87d93b6f2b7d3432254565c75c1ef73e54f90: Status 404 returned error can't find the container with id a09fef78107215983939a82a37e87d93b6f2b7d3432254565c75c1ef73e54f90
Sep 15 20:32:09.718214 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:09.718174    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr" event={"ID":"2b77ce5f-d129-4eb5-966c-aae7ad0c1d04","Type":"ContainerStarted","Data":"19a6ac4951b04e24475f0dcb0d4c2bf6b8d70819978193cfa22111f3fe98cd7e"}
Sep 15 20:32:09.718616 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:09.718225    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr" event={"ID":"2b77ce5f-d129-4eb5-966c-aae7ad0c1d04","Type":"ContainerStarted","Data":"a09fef78107215983939a82a37e87d93b6f2b7d3432254565c75c1ef73e54f90"}
Sep 15 20:32:09.734098 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:09.734047    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr" podStartSLOduration=1.734032461 podStartE2EDuration="1.734032461s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:32:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:32:09.732721508 +0000 UTC m=+1644.221185983" watchObservedRunningTime="2026-09-15 20:32:09.734032461 +0000 UTC m=+1644.222496936"
Sep 15 20:32:13.063541 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:13.063484    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:32:16.516019 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:16.515972    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:32:16.785127 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:16.785052    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:32:18.934624 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:18.934592    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr"
Sep 15 20:32:18.935056 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:18.934843    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr"
Sep 15 20:32:18.947240 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:18.947221    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr"
Sep 15 20:32:19.782146 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:19.782114    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr"
Sep 15 20:32:21.250710 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:21.250680    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:32:33.818388 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:33.818302    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:32:37.044378 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:37.044342    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:32:37.793411 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:37.793379    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:32:44.047018 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:44.046970    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:32:44.147050 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:44.147009    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:32:44.405418 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:44.405340    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:32:44.605850 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:44.605818    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:32:47.316725 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:47.316685    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:32:47.929299 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:47.929249    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:32:48.021028 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:48.021000    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:32:48.878258 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:48.878226    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:32:49.626099 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:49.626060    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:32:50.344711 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:50.344676    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:32:50.973516 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:50.973489    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:32:51.594739 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:51.594701    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:32:51.886084 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:51.886004    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:32:52.472210 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:52.472178    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:32:52.742343 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:52.742264    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:32:52.800236 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:52.800180    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:32:53.035269 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:53.035179    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:32:53.115141 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:53.115108    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:32:53.116598 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:53.116576    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a8e967-84f8899bc8-crskq"]
Sep 15 20:32:53.120358 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:53.120326    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a8e967-84f8899bc8-crskq"
Sep 15 20:32:53.122643 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:53.122626    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-a8e967\""
Sep 15 20:32:53.128738 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:53.128718    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a8e967-84f8899bc8-crskq"]
Sep 15 20:32:53.158506 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:53.158477    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a8e967-5587c45687-hbckj"]
Sep 15 20:32:53.162224 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:53.162204    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-a8e967-5587c45687-hbckj"
Sep 15 20:32:53.164381 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:53.164364    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-a8e967-dockercfg-hw8jq\""
Sep 15 20:32:53.172666 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:53.172645    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a8e967-5587c45687-hbckj"]
Sep 15 20:32:53.210461 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:53.210435    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcn6g\" (UniqueName: \"kubernetes.io/projected/9b2e7762-3a06-401a-bb79-8ee04fbfdbef-kube-api-access-jcn6g\") pod \"payload-processing-e2e-api-a-a8e967-5587c45687-hbckj\" (UID: \"9b2e7762-3a06-401a-bb79-8ee04fbfdbef\") " pod="openshift-ingress/payload-processing-e2e-api-a-a8e967-5587c45687-hbckj"
Sep 15 20:32:53.210564 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:53.210471    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9212e15-6b95-4271-8d8c-df2627d16662-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-a8e967-84f8899bc8-crskq\" (UID: \"b9212e15-6b95-4271-8d8c-df2627d16662\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a8e967-84f8899bc8-crskq"
Sep 15 20:32:53.210564 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:53.210492    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b2e7762-3a06-401a-bb79-8ee04fbfdbef-plugins-config-volume\") pod \"payload-processing-e2e-api-a-a8e967-5587c45687-hbckj\" (UID: \"9b2e7762-3a06-401a-bb79-8ee04fbfdbef\") " pod="openshift-ingress/payload-processing-e2e-api-a-a8e967-5587c45687-hbckj"
Sep 15 20:32:53.210674 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:53.210655    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmfdj\" (UniqueName: \"kubernetes.io/projected/b9212e15-6b95-4271-8d8c-df2627d16662-kube-api-access-fmfdj\") pod \"payload-pre-processing-e2e-api-a-a8e967-84f8899bc8-crskq\" (UID: \"b9212e15-6b95-4271-8d8c-df2627d16662\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a8e967-84f8899bc8-crskq"
Sep 15 20:32:53.311431 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:53.311368    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9212e15-6b95-4271-8d8c-df2627d16662-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-a8e967-84f8899bc8-crskq\" (UID: \"b9212e15-6b95-4271-8d8c-df2627d16662\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a8e967-84f8899bc8-crskq"
Sep 15 20:32:53.311431 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:53.311403    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b2e7762-3a06-401a-bb79-8ee04fbfdbef-plugins-config-volume\") pod \"payload-processing-e2e-api-a-a8e967-5587c45687-hbckj\" (UID: \"9b2e7762-3a06-401a-bb79-8ee04fbfdbef\") " pod="openshift-ingress/payload-processing-e2e-api-a-a8e967-5587c45687-hbckj"
Sep 15 20:32:53.311601 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:53.311485    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmfdj\" (UniqueName: \"kubernetes.io/projected/b9212e15-6b95-4271-8d8c-df2627d16662-kube-api-access-fmfdj\") pod \"payload-pre-processing-e2e-api-a-a8e967-84f8899bc8-crskq\" (UID: \"b9212e15-6b95-4271-8d8c-df2627d16662\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a8e967-84f8899bc8-crskq"
Sep 15 20:32:53.311655 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:53.311607    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcn6g\" (UniqueName: \"kubernetes.io/projected/9b2e7762-3a06-401a-bb79-8ee04fbfdbef-kube-api-access-jcn6g\") pod \"payload-processing-e2e-api-a-a8e967-5587c45687-hbckj\" (UID: \"9b2e7762-3a06-401a-bb79-8ee04fbfdbef\") " pod="openshift-ingress/payload-processing-e2e-api-a-a8e967-5587c45687-hbckj"
Sep 15 20:32:53.312146 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:53.312128    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9212e15-6b95-4271-8d8c-df2627d16662-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-a8e967-84f8899bc8-crskq\" (UID: \"b9212e15-6b95-4271-8d8c-df2627d16662\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a8e967-84f8899bc8-crskq"
Sep 15 20:32:53.312200 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:53.312180    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b2e7762-3a06-401a-bb79-8ee04fbfdbef-plugins-config-volume\") pod \"payload-processing-e2e-api-a-a8e967-5587c45687-hbckj\" (UID: \"9b2e7762-3a06-401a-bb79-8ee04fbfdbef\") " pod="openshift-ingress/payload-processing-e2e-api-a-a8e967-5587c45687-hbckj"
Sep 15 20:32:53.319716 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:53.319694    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmfdj\" (UniqueName: \"kubernetes.io/projected/b9212e15-6b95-4271-8d8c-df2627d16662-kube-api-access-fmfdj\") pod \"payload-pre-processing-e2e-api-a-a8e967-84f8899bc8-crskq\" (UID: \"b9212e15-6b95-4271-8d8c-df2627d16662\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a8e967-84f8899bc8-crskq"
Sep 15 20:32:53.319826 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:53.319779    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcn6g\" (UniqueName: \"kubernetes.io/projected/9b2e7762-3a06-401a-bb79-8ee04fbfdbef-kube-api-access-jcn6g\") pod \"payload-processing-e2e-api-a-a8e967-5587c45687-hbckj\" (UID: \"9b2e7762-3a06-401a-bb79-8ee04fbfdbef\") " pod="openshift-ingress/payload-processing-e2e-api-a-a8e967-5587c45687-hbckj"
Sep 15 20:32:53.431851 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:53.431830    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a8e967-84f8899bc8-crskq"
Sep 15 20:32:53.473098 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:53.473072    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-a8e967-5587c45687-hbckj"
Sep 15 20:32:53.706313 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:53.706288    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a8e967-84f8899bc8-crskq"]
Sep 15 20:32:53.707705 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:32:53.707681    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9212e15_6b95_4271_8d8c_df2627d16662.slice/crio-ff48ea7eb3b4eda770242d4508e5c9da3f0f5188d8caf5ba0397206eabe398ba WatchSource:0}: Error finding container ff48ea7eb3b4eda770242d4508e5c9da3f0f5188d8caf5ba0397206eabe398ba: Status 404 returned error can't find the container with id ff48ea7eb3b4eda770242d4508e5c9da3f0f5188d8caf5ba0397206eabe398ba
Sep 15 20:32:53.758485 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:53.758462    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a8e967-5587c45687-hbckj"]
Sep 15 20:32:53.760013 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:32:53.759989    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b2e7762_3a06_401a_bb79_8ee04fbfdbef.slice/crio-797cae4fa21e0ffa70038f85119fd9f17c651af651431fadde0342f119d1a85a WatchSource:0}: Error finding container 797cae4fa21e0ffa70038f85119fd9f17c651af651431fadde0342f119d1a85a: Status 404 returned error can't find the container with id 797cae4fa21e0ffa70038f85119fd9f17c651af651431fadde0342f119d1a85a
Sep 15 20:32:53.956963 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:53.956855    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a8e967-84f8899bc8-crskq" event={"ID":"b9212e15-6b95-4271-8d8c-df2627d16662","Type":"ContainerStarted","Data":"b6b5423bf741154f4c7bef92866764c71b1d17af22d19a535d1b9a00d0719ec1"}
Sep 15 20:32:53.957130 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:53.957067    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a8e967-84f8899bc8-crskq"
Sep 15 20:32:53.957130 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:53.957093    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a8e967-84f8899bc8-crskq" event={"ID":"b9212e15-6b95-4271-8d8c-df2627d16662","Type":"ContainerStarted","Data":"ff48ea7eb3b4eda770242d4508e5c9da3f0f5188d8caf5ba0397206eabe398ba"}
Sep 15 20:32:53.958559 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:53.958536    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-a8e967-5587c45687-hbckj" event={"ID":"9b2e7762-3a06-401a-bb79-8ee04fbfdbef","Type":"ContainerStarted","Data":"51c9ca26b91a3d3dd38b81355e82e3ae98789b1010983a4660168eb1f32d9fcb"}
Sep 15 20:32:53.958694 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:53.958582    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-a8e967-5587c45687-hbckj" event={"ID":"9b2e7762-3a06-401a-bb79-8ee04fbfdbef","Type":"ContainerStarted","Data":"797cae4fa21e0ffa70038f85119fd9f17c651af651431fadde0342f119d1a85a"}
Sep 15 20:32:53.958694 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:53.958609    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-a8e967-5587c45687-hbckj"
Sep 15 20:32:53.972837 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:53.972792    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a8e967-84f8899bc8-crskq" podStartSLOduration=0.972776487 podStartE2EDuration="972.776487ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:32:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:32:53.970794422 +0000 UTC m=+1688.459258914" watchObservedRunningTime="2026-09-15 20:32:53.972776487 +0000 UTC m=+1688.461240963"
Sep 15 20:32:53.986776 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:32:53.986734    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-a8e967-5587c45687-hbckj" podStartSLOduration=0.986720786 podStartE2EDuration="986.720786ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:32:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:32:53.984712395 +0000 UTC m=+1688.473176883" watchObservedRunningTime="2026-09-15 20:32:53.986720786 +0000 UTC m=+1688.475185261"
Sep 15 20:33:04.967133 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:04.967089    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-a8e967-5587c45687-hbckj"
Sep 15 20:33:04.967494 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:04.967185    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a8e967-84f8899bc8-crskq"
Sep 15 20:33:14.498611 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:14.498508    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:15.283534 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:15.283475    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:15.986756 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:15.986716    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:16.647502 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:16.647459    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:17.468391 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:17.468352    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:18.124288 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:18.124255    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:18.340254 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:18.340213    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:18.757946 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:18.757887    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:18.855555 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:18.855523    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:19.060807 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:19.060703    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:19.162523 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:19.162491    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:19.222082 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:19.222048    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:19.324833 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:19.324751    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:19.529895 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:19.529861    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:21.661248 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:21.661211    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:22.344573 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:22.344534    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:22.436893 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:22.436855    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:22.756260 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:22.756230    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:23.431563 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:23.431525    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:24.254847 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:24.254797    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:24.876795 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:24.876758    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:25.491222 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:25.491190    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:26.166186 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:26.166147    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:26.780395 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:26.780361    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:27.569373 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:27.569330    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:27.626078 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:27.626043    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:27.851200 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:27.851108    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:28.021564 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:28.021532    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:28.112727 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:28.112650    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:28.170252 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:28.170214    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-df0188-75948b989f-rmmgq"]
Sep 15 20:33:28.174942 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:28.174910    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-df0188-75948b989f-rmmgq"
Sep 15 20:33:28.177782 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:28.177761    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-df0188-dockercfg-bqm9q\""
Sep 15 20:33:28.177979 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:28.177954    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-df0188\""
Sep 15 20:33:28.180355 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:28.180332    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-df0188-75948b989f-rmmgq"]
Sep 15 20:33:28.254535 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:28.254492    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c456c132-4bb9-4325-a972-ccc0057ef664-plugins-config-volume\") pod \"payload-processing-e2e-api-b-df0188-75948b989f-rmmgq\" (UID: \"c456c132-4bb9-4325-a972-ccc0057ef664\") " pod="openshift-ingress/payload-processing-e2e-api-b-df0188-75948b989f-rmmgq"
Sep 15 20:33:28.254706 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:28.254567    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2c2j\" (UniqueName: \"kubernetes.io/projected/c456c132-4bb9-4325-a972-ccc0057ef664-kube-api-access-h2c2j\") pod \"payload-processing-e2e-api-b-df0188-75948b989f-rmmgq\" (UID: \"c456c132-4bb9-4325-a972-ccc0057ef664\") " pod="openshift-ingress/payload-processing-e2e-api-b-df0188-75948b989f-rmmgq"
Sep 15 20:33:28.355766 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:28.355735    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c456c132-4bb9-4325-a972-ccc0057ef664-plugins-config-volume\") pod \"payload-processing-e2e-api-b-df0188-75948b989f-rmmgq\" (UID: \"c456c132-4bb9-4325-a972-ccc0057ef664\") " pod="openshift-ingress/payload-processing-e2e-api-b-df0188-75948b989f-rmmgq"
Sep 15 20:33:28.355966 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:28.355786    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2c2j\" (UniqueName: \"kubernetes.io/projected/c456c132-4bb9-4325-a972-ccc0057ef664-kube-api-access-h2c2j\") pod \"payload-processing-e2e-api-b-df0188-75948b989f-rmmgq\" (UID: \"c456c132-4bb9-4325-a972-ccc0057ef664\") " pod="openshift-ingress/payload-processing-e2e-api-b-df0188-75948b989f-rmmgq"
Sep 15 20:33:28.356501 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:28.356474    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c456c132-4bb9-4325-a972-ccc0057ef664-plugins-config-volume\") pod \"payload-processing-e2e-api-b-df0188-75948b989f-rmmgq\" (UID: \"c456c132-4bb9-4325-a972-ccc0057ef664\") " pod="openshift-ingress/payload-processing-e2e-api-b-df0188-75948b989f-rmmgq"
Sep 15 20:33:28.363630 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:28.363582    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2c2j\" (UniqueName: \"kubernetes.io/projected/c456c132-4bb9-4325-a972-ccc0057ef664-kube-api-access-h2c2j\") pod \"payload-processing-e2e-api-b-df0188-75948b989f-rmmgq\" (UID: \"c456c132-4bb9-4325-a972-ccc0057ef664\") " pod="openshift-ingress/payload-processing-e2e-api-b-df0188-75948b989f-rmmgq"
Sep 15 20:33:28.486753 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:28.486724    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-df0188-75948b989f-rmmgq"
Sep 15 20:33:28.756575 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:28.756545    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-df0188-75948b989f-rmmgq"]
Sep 15 20:33:28.757285 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:33:28.757260    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc456c132_4bb9_4325_a972_ccc0057ef664.slice/crio-4819b8a3ea2f7fdb322f94a05d1f37b11b8cd7d764afeaddf3d139ae26a7c397 WatchSource:0}: Error finding container 4819b8a3ea2f7fdb322f94a05d1f37b11b8cd7d764afeaddf3d139ae26a7c397: Status 404 returned error can't find the container with id 4819b8a3ea2f7fdb322f94a05d1f37b11b8cd7d764afeaddf3d139ae26a7c397
Sep 15 20:33:29.161943 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:29.161832    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-df0188-75948b989f-rmmgq" event={"ID":"c456c132-4bb9-4325-a972-ccc0057ef664","Type":"ContainerStarted","Data":"9aa3054486bd6351de25299e51bb1e47673e1cb009796a12ebde501fd397abf6"}
Sep 15 20:33:29.161943 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:29.161893    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-df0188-75948b989f-rmmgq" event={"ID":"c456c132-4bb9-4325-a972-ccc0057ef664","Type":"ContainerStarted","Data":"4819b8a3ea2f7fdb322f94a05d1f37b11b8cd7d764afeaddf3d139ae26a7c397"}
Sep 15 20:33:29.161943 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:29.161939    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-df0188-75948b989f-rmmgq"
Sep 15 20:33:29.179186 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:29.179137    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-df0188-75948b989f-rmmgq" podStartSLOduration=1.1791228280000001 podStartE2EDuration="1.179122828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:33:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:33:29.176763823 +0000 UTC m=+1723.665228300" watchObservedRunningTime="2026-09-15 20:33:29.179122828 +0000 UTC m=+1723.667587320"
Sep 15 20:33:40.170696 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:40.170665    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-df0188-75948b989f-rmmgq"
Sep 15 20:33:49.557747 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:49.557709    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:50.190138 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:50.190079    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:50.801468 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:50.801433    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:51.570058 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:51.570022    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:52.250420 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:52.250386    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:52.810234 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:52.810197    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:53.042168 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:53.042137    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:53.142862 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:53.142782    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:53.981335 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:53.981298    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:54.685114 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:54.685084    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:55.378082 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:55.378052    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:57.054545 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:57.054510    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:57.886413 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:57.886371    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:58.557129 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:58.557034    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:59.192884 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:59.192851    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:33:59.972431 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:33:59.972395    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:00.620693 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:00.620661    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:01.460261 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:01.460226    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:02.146035 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:02.146001    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:02.768301 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:02.768267    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:03.473471 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:03.473441    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:12.094120 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.094086    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:12.119148 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.119116    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a8e967-5587c45687-hbckj"]
Sep 15 20:34:12.119523 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.119355    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-a8e967-5587c45687-hbckj" podUID="9b2e7762-3a06-401a-bb79-8ee04fbfdbef" containerName="payload-processing" containerID="cri-o://51c9ca26b91a3d3dd38b81355e82e3ae98789b1010983a4660168eb1f32d9fcb" gracePeriod=30
Sep 15 20:34:12.127839 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.127810    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a8e967-84f8899bc8-crskq"]
Sep 15 20:34:12.128101 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.128074    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a8e967-84f8899bc8-crskq" podUID="b9212e15-6b95-4271-8d8c-df2627d16662" containerName="payload-pre-processing" containerID="cri-o://b6b5423bf741154f4c7bef92866764c71b1d17af22d19a535d1b9a00d0719ec1" gracePeriod=30
Sep 15 20:34:12.156033 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.156003    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:12.319655 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.319619    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:12.399284 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.399258    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:12.413540 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.413520    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-a8e967-5587c45687-hbckj"
Sep 15 20:34:12.416886 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.416855    2576 generic.go:356] "Generic (PLEG): container finished" podID="9b2e7762-3a06-401a-bb79-8ee04fbfdbef" containerID="51c9ca26b91a3d3dd38b81355e82e3ae98789b1010983a4660168eb1f32d9fcb" exitCode=0
Sep 15 20:34:12.417007 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.416887    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-a8e967-5587c45687-hbckj"
Sep 15 20:34:12.417007 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.416887    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-a8e967-5587c45687-hbckj" event={"ID":"9b2e7762-3a06-401a-bb79-8ee04fbfdbef","Type":"ContainerDied","Data":"51c9ca26b91a3d3dd38b81355e82e3ae98789b1010983a4660168eb1f32d9fcb"}
Sep 15 20:34:12.417007 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.416977    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-a8e967-5587c45687-hbckj" event={"ID":"9b2e7762-3a06-401a-bb79-8ee04fbfdbef","Type":"ContainerDied","Data":"797cae4fa21e0ffa70038f85119fd9f17c651af651431fadde0342f119d1a85a"}
Sep 15 20:34:12.417007 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.417001    2576 scope.go:117] "RemoveContainer" containerID="51c9ca26b91a3d3dd38b81355e82e3ae98789b1010983a4660168eb1f32d9fcb"
Sep 15 20:34:12.417196 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.417071    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a8e967-84f8899bc8-crskq"
Sep 15 20:34:12.418264 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.418246    2576 generic.go:356] "Generic (PLEG): container finished" podID="b9212e15-6b95-4271-8d8c-df2627d16662" containerID="b6b5423bf741154f4c7bef92866764c71b1d17af22d19a535d1b9a00d0719ec1" exitCode=0
Sep 15 20:34:12.418340 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.418288    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a8e967-84f8899bc8-crskq" event={"ID":"b9212e15-6b95-4271-8d8c-df2627d16662","Type":"ContainerDied","Data":"b6b5423bf741154f4c7bef92866764c71b1d17af22d19a535d1b9a00d0719ec1"}
Sep 15 20:34:12.418340 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.418327    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a8e967-84f8899bc8-crskq" event={"ID":"b9212e15-6b95-4271-8d8c-df2627d16662","Type":"ContainerDied","Data":"ff48ea7eb3b4eda770242d4508e5c9da3f0f5188d8caf5ba0397206eabe398ba"}
Sep 15 20:34:12.426222 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.426207    2576 scope.go:117] "RemoveContainer" containerID="51c9ca26b91a3d3dd38b81355e82e3ae98789b1010983a4660168eb1f32d9fcb"
Sep 15 20:34:12.426483 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:34:12.426465    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51c9ca26b91a3d3dd38b81355e82e3ae98789b1010983a4660168eb1f32d9fcb\": container with ID starting with 51c9ca26b91a3d3dd38b81355e82e3ae98789b1010983a4660168eb1f32d9fcb not found: ID does not exist" containerID="51c9ca26b91a3d3dd38b81355e82e3ae98789b1010983a4660168eb1f32d9fcb"
Sep 15 20:34:12.426538 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.426491    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51c9ca26b91a3d3dd38b81355e82e3ae98789b1010983a4660168eb1f32d9fcb"} err="failed to get container status \"51c9ca26b91a3d3dd38b81355e82e3ae98789b1010983a4660168eb1f32d9fcb\": rpc error: code = NotFound desc = could not find container \"51c9ca26b91a3d3dd38b81355e82e3ae98789b1010983a4660168eb1f32d9fcb\": container with ID starting with 51c9ca26b91a3d3dd38b81355e82e3ae98789b1010983a4660168eb1f32d9fcb not found: ID does not exist"
Sep 15 20:34:12.426538 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.426513    2576 scope.go:117] "RemoveContainer" containerID="b6b5423bf741154f4c7bef92866764c71b1d17af22d19a535d1b9a00d0719ec1"
Sep 15 20:34:12.436638 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.436622    2576 scope.go:117] "RemoveContainer" containerID="b6b5423bf741154f4c7bef92866764c71b1d17af22d19a535d1b9a00d0719ec1"
Sep 15 20:34:12.436956 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:34:12.436907    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6b5423bf741154f4c7bef92866764c71b1d17af22d19a535d1b9a00d0719ec1\": container with ID starting with b6b5423bf741154f4c7bef92866764c71b1d17af22d19a535d1b9a00d0719ec1 not found: ID does not exist" containerID="b6b5423bf741154f4c7bef92866764c71b1d17af22d19a535d1b9a00d0719ec1"
Sep 15 20:34:12.437075 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.436962    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6b5423bf741154f4c7bef92866764c71b1d17af22d19a535d1b9a00d0719ec1"} err="failed to get container status \"b6b5423bf741154f4c7bef92866764c71b1d17af22d19a535d1b9a00d0719ec1\": rpc error: code = NotFound desc = could not find container \"b6b5423bf741154f4c7bef92866764c71b1d17af22d19a535d1b9a00d0719ec1\": container with ID starting with b6b5423bf741154f4c7bef92866764c71b1d17af22d19a535d1b9a00d0719ec1 not found: ID does not exist"
Sep 15 20:34:12.465724 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.465700    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:12.487695 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.487668    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmfdj\" (UniqueName: \"kubernetes.io/projected/b9212e15-6b95-4271-8d8c-df2627d16662-kube-api-access-fmfdj\") pod \"b9212e15-6b95-4271-8d8c-df2627d16662\" (UID: \"b9212e15-6b95-4271-8d8c-df2627d16662\") "
Sep 15 20:34:12.487821 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.487728    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b2e7762-3a06-401a-bb79-8ee04fbfdbef-plugins-config-volume\") pod \"9b2e7762-3a06-401a-bb79-8ee04fbfdbef\" (UID: \"9b2e7762-3a06-401a-bb79-8ee04fbfdbef\") "
Sep 15 20:34:12.487821 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.487769    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jcn6g\" (UniqueName: \"kubernetes.io/projected/9b2e7762-3a06-401a-bb79-8ee04fbfdbef-kube-api-access-jcn6g\") pod \"9b2e7762-3a06-401a-bb79-8ee04fbfdbef\" (UID: \"9b2e7762-3a06-401a-bb79-8ee04fbfdbef\") "
Sep 15 20:34:12.487821 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.487803    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9212e15-6b95-4271-8d8c-df2627d16662-plugins-config-volume\") pod \"b9212e15-6b95-4271-8d8c-df2627d16662\" (UID: \"b9212e15-6b95-4271-8d8c-df2627d16662\") "
Sep 15 20:34:12.488179 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.488142    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b2e7762-3a06-401a-bb79-8ee04fbfdbef-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9b2e7762-3a06-401a-bb79-8ee04fbfdbef" (UID: "9b2e7762-3a06-401a-bb79-8ee04fbfdbef"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:34:12.488287 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.488220    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9212e15-6b95-4271-8d8c-df2627d16662-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b9212e15-6b95-4271-8d8c-df2627d16662" (UID: "b9212e15-6b95-4271-8d8c-df2627d16662"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:34:12.489851 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.489825    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b2e7762-3a06-401a-bb79-8ee04fbfdbef-kube-api-access-jcn6g" (OuterVolumeSpecName: "kube-api-access-jcn6g") pod "9b2e7762-3a06-401a-bb79-8ee04fbfdbef" (UID: "9b2e7762-3a06-401a-bb79-8ee04fbfdbef"). InnerVolumeSpecName "kube-api-access-jcn6g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:34:12.489976 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.489854    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9212e15-6b95-4271-8d8c-df2627d16662-kube-api-access-fmfdj" (OuterVolumeSpecName: "kube-api-access-fmfdj") pod "b9212e15-6b95-4271-8d8c-df2627d16662" (UID: "b9212e15-6b95-4271-8d8c-df2627d16662"). InnerVolumeSpecName "kube-api-access-fmfdj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:34:12.589132 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.589092    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmfdj\" (UniqueName: \"kubernetes.io/projected/b9212e15-6b95-4271-8d8c-df2627d16662-kube-api-access-fmfdj\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:34:12.589132 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.589129    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b2e7762-3a06-401a-bb79-8ee04fbfdbef-plugins-config-volume\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:34:12.589132 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.589139    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcn6g\" (UniqueName: \"kubernetes.io/projected/9b2e7762-3a06-401a-bb79-8ee04fbfdbef-kube-api-access-jcn6g\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:34:12.589330 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.589148    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9212e15-6b95-4271-8d8c-df2627d16662-plugins-config-volume\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:34:12.738535 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.738504    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a8e967-5587c45687-hbckj"]
Sep 15 20:34:12.743494 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.743461    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a8e967-5587c45687-hbckj"]
Sep 15 20:34:12.801443 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.801408    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:12.901540 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:12.901505    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:13.425992 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:13.425894    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a8e967-84f8899bc8-crskq"
Sep 15 20:34:13.447039 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:13.447006    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a8e967-84f8899bc8-crskq"]
Sep 15 20:34:13.449770 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:13.449730    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a8e967-84f8899bc8-crskq"]
Sep 15 20:34:13.647729 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:13.647695    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:14.144708 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:14.144677    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b2e7762-3a06-401a-bb79-8ee04fbfdbef" path="/var/lib/kubelet/pods/9b2e7762-3a06-401a-bb79-8ee04fbfdbef/volumes"
Sep 15 20:34:14.145076 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:14.145060    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9212e15-6b95-4271-8d8c-df2627d16662" path="/var/lib/kubelet/pods/b9212e15-6b95-4271-8d8c-df2627d16662/volumes"
Sep 15 20:34:14.474558 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:14.474531    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:15.197884 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:15.197849    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:15.947156 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:15.947119    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:16.684844 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:16.684811    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:17.460018 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:17.459979    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:18.260506 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:18.260467    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:19.086741 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:19.086707    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:19.152593 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:19.152559    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:19.405902 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:19.405825    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:19.833337 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:19.833304    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-df0188-2kd57"]
Sep 15 20:34:19.833775 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:19.833762    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9212e15-6b95-4271-8d8c-df2627d16662" containerName="payload-pre-processing"
Sep 15 20:34:19.833819 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:19.833776    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9212e15-6b95-4271-8d8c-df2627d16662" containerName="payload-pre-processing"
Sep 15 20:34:19.833819 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:19.833789    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b2e7762-3a06-401a-bb79-8ee04fbfdbef" containerName="payload-processing"
Sep 15 20:34:19.833819 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:19.833795    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b2e7762-3a06-401a-bb79-8ee04fbfdbef" containerName="payload-processing"
Sep 15 20:34:19.833935 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:19.833878    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9212e15-6b95-4271-8d8c-df2627d16662" containerName="payload-pre-processing"
Sep 15 20:34:19.833935 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:19.833888    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b2e7762-3a06-401a-bb79-8ee04fbfdbef" containerName="payload-processing"
Sep 15 20:34:19.837110 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:19.837091    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-df0188-2kd57"
Sep 15 20:34:19.839580 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:19.839556    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 20:34:19.839707 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:19.839640    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r5lpt\""
Sep 15 20:34:19.845546 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:19.845519    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-df0188-2kd57"]
Sep 15 20:34:19.964599 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:19.964562    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf5e104a-6fca-47cf-a949-5fb3f21e142e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-df0188-2kd57\" (UID: \"cf5e104a-6fca-47cf-a949-5fb3f21e142e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-df0188-2kd57"
Sep 15 20:34:20.042373 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:20.042333    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:20.065635 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:20.065600    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf5e104a-6fca-47cf-a949-5fb3f21e142e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-df0188-2kd57\" (UID: \"cf5e104a-6fca-47cf-a949-5fb3f21e142e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-df0188-2kd57"
Sep 15 20:34:20.066268 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:20.066250    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf5e104a-6fca-47cf-a949-5fb3f21e142e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-df0188-2kd57\" (UID: \"cf5e104a-6fca-47cf-a949-5fb3f21e142e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-df0188-2kd57"
Sep 15 20:34:20.148704 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:20.148627    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-df0188-2kd57"
Sep 15 20:34:20.423302 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:20.423277    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-df0188-2kd57"]
Sep 15 20:34:20.424450 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:34:20.424422    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf5e104a_6fca_47cf_a949_5fb3f21e142e.slice/crio-d35a52199aa0b2d3793288f2db7b1080cdca4ffdf244a0f1ad4bc4fa7b0c09c5 WatchSource:0}: Error finding container d35a52199aa0b2d3793288f2db7b1080cdca4ffdf244a0f1ad4bc4fa7b0c09c5: Status 404 returned error can't find the container with id d35a52199aa0b2d3793288f2db7b1080cdca4ffdf244a0f1ad4bc4fa7b0c09c5
Sep 15 20:34:20.467654 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:20.467622    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-df0188-2kd57" event={"ID":"cf5e104a-6fca-47cf-a949-5fb3f21e142e","Type":"ContainerStarted","Data":"d35a52199aa0b2d3793288f2db7b1080cdca4ffdf244a0f1ad4bc4fa7b0c09c5"}
Sep 15 20:34:20.851386 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:20.851352    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:21.474637 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:21.474607    2576 generic.go:356] "Generic (PLEG): container finished" podID="cf5e104a-6fca-47cf-a949-5fb3f21e142e" containerID="f90bfe49bb3acb6acbfcdb60cab9375fb1290d2df7a9c291cd841655e3c31dd1" exitCode=0
Sep 15 20:34:21.475104 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:21.474664    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-df0188-2kd57" event={"ID":"cf5e104a-6fca-47cf-a949-5fb3f21e142e","Type":"ContainerDied","Data":"f90bfe49bb3acb6acbfcdb60cab9375fb1290d2df7a9c291cd841655e3c31dd1"}
Sep 15 20:34:21.622055 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:21.622015    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:22.293095 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:22.293066    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:22.614206 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:22.614180    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-df0188-2kd57"
Sep 15 20:34:22.794677 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:22.794639    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf5e104a-6fca-47cf-a949-5fb3f21e142e-maas-api-service-ca\") pod \"cf5e104a-6fca-47cf-a949-5fb3f21e142e\" (UID: \"cf5e104a-6fca-47cf-a949-5fb3f21e142e\") "
Sep 15 20:34:22.795012 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:22.794990    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf5e104a-6fca-47cf-a949-5fb3f21e142e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cf5e104a-6fca-47cf-a949-5fb3f21e142e" (UID: "cf5e104a-6fca-47cf-a949-5fb3f21e142e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:34:22.895987 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:22.895890    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf5e104a-6fca-47cf-a949-5fb3f21e142e-maas-api-service-ca\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:34:22.922663 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:22.922623    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:23.487238 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:23.487205    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-df0188-2kd57"
Sep 15 20:34:23.487238 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:23.487231    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-df0188-2kd57" event={"ID":"cf5e104a-6fca-47cf-a949-5fb3f21e142e","Type":"ContainerDied","Data":"d35a52199aa0b2d3793288f2db7b1080cdca4ffdf244a0f1ad4bc4fa7b0c09c5"}
Sep 15 20:34:23.487443 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:23.487273    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d35a52199aa0b2d3793288f2db7b1080cdca4ffdf244a0f1ad4bc4fa7b0c09c5"
Sep 15 20:34:23.763153 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:23.763064    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:24.391591 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:24.391556    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:25.047576 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:25.047546    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:25.772968 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:25.772913    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:35.260908 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:35.260877    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:35.289723 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:35.289690    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-df0188-75948b989f-rmmgq"]
Sep 15 20:34:35.290523 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:35.290014    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-df0188-75948b989f-rmmgq" podUID="c456c132-4bb9-4325-a972-ccc0057ef664" containerName="payload-processing" containerID="cri-o://9aa3054486bd6351de25299e51bb1e47673e1cb009796a12ebde501fd397abf6" gracePeriod=30
Sep 15 20:34:35.361494 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:35.361461    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:35.439971 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:35.439909    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-df0188-2kd57"]
Sep 15 20:34:35.442274 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:35.442241    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:35.443029 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:35.443010    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-df0188-2kd57"]
Sep 15 20:34:35.497602 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:35.497574    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:35.549189 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:35.549163    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-df0188-75948b989f-rmmgq"
Sep 15 20:34:35.567970 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:35.567937    2576 generic.go:356] "Generic (PLEG): container finished" podID="c456c132-4bb9-4325-a972-ccc0057ef664" containerID="9aa3054486bd6351de25299e51bb1e47673e1cb009796a12ebde501fd397abf6" exitCode=0
Sep 15 20:34:35.567970 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:35.567956    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-df0188-75948b989f-rmmgq" event={"ID":"c456c132-4bb9-4325-a972-ccc0057ef664","Type":"ContainerDied","Data":"9aa3054486bd6351de25299e51bb1e47673e1cb009796a12ebde501fd397abf6"}
Sep 15 20:34:35.567970 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:35.567966    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-df0188-75948b989f-rmmgq"
Sep 15 20:34:35.568199 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:35.568010    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-df0188-75948b989f-rmmgq" event={"ID":"c456c132-4bb9-4325-a972-ccc0057ef664","Type":"ContainerDied","Data":"4819b8a3ea2f7fdb322f94a05d1f37b11b8cd7d764afeaddf3d139ae26a7c397"}
Sep 15 20:34:35.568199 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:35.568038    2576 scope.go:117] "RemoveContainer" containerID="9aa3054486bd6351de25299e51bb1e47673e1cb009796a12ebde501fd397abf6"
Sep 15 20:34:35.578336 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:35.578317    2576 scope.go:117] "RemoveContainer" containerID="9aa3054486bd6351de25299e51bb1e47673e1cb009796a12ebde501fd397abf6"
Sep 15 20:34:35.578603 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:34:35.578587    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9aa3054486bd6351de25299e51bb1e47673e1cb009796a12ebde501fd397abf6\": container with ID starting with 9aa3054486bd6351de25299e51bb1e47673e1cb009796a12ebde501fd397abf6 not found: ID does not exist" containerID="9aa3054486bd6351de25299e51bb1e47673e1cb009796a12ebde501fd397abf6"
Sep 15 20:34:35.578665 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:35.578611    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9aa3054486bd6351de25299e51bb1e47673e1cb009796a12ebde501fd397abf6"} err="failed to get container status \"9aa3054486bd6351de25299e51bb1e47673e1cb009796a12ebde501fd397abf6\": rpc error: code = NotFound desc = could not find container \"9aa3054486bd6351de25299e51bb1e47673e1cb009796a12ebde501fd397abf6\": container with ID starting with 9aa3054486bd6351de25299e51bb1e47673e1cb009796a12ebde501fd397abf6 not found: ID does not exist"
Sep 15 20:34:35.620716 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:35.620693    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c456c132-4bb9-4325-a972-ccc0057ef664-plugins-config-volume\") pod \"c456c132-4bb9-4325-a972-ccc0057ef664\" (UID: \"c456c132-4bb9-4325-a972-ccc0057ef664\") "
Sep 15 20:34:35.620828 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:35.620817    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2c2j\" (UniqueName: \"kubernetes.io/projected/c456c132-4bb9-4325-a972-ccc0057ef664-kube-api-access-h2c2j\") pod \"c456c132-4bb9-4325-a972-ccc0057ef664\" (UID: \"c456c132-4bb9-4325-a972-ccc0057ef664\") "
Sep 15 20:34:35.621060 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:35.621039    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c456c132-4bb9-4325-a972-ccc0057ef664-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c456c132-4bb9-4325-a972-ccc0057ef664" (UID: "c456c132-4bb9-4325-a972-ccc0057ef664"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:34:35.622739 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:35.622720    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c456c132-4bb9-4325-a972-ccc0057ef664-kube-api-access-h2c2j" (OuterVolumeSpecName: "kube-api-access-h2c2j") pod "c456c132-4bb9-4325-a972-ccc0057ef664" (UID: "c456c132-4bb9-4325-a972-ccc0057ef664"). InnerVolumeSpecName "kube-api-access-h2c2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:34:35.722356 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:35.722332    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c456c132-4bb9-4325-a972-ccc0057ef664-plugins-config-volume\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:34:35.722356 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:35.722355    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2c2j\" (UniqueName: \"kubernetes.io/projected/c456c132-4bb9-4325-a972-ccc0057ef664-kube-api-access-h2c2j\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:34:35.840381 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:35.840305    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:35.889414 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:35.889381    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-df0188-75948b989f-rmmgq"]
Sep 15 20:34:35.894149 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:35.894121    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-df0188-75948b989f-rmmgq"]
Sep 15 20:34:35.898290 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:35.898263    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:36.150979 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:36.150883    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c456c132-4bb9-4325-a972-ccc0057ef664" path="/var/lib/kubelet/pods/c456c132-4bb9-4325-a972-ccc0057ef664/volumes"
Sep 15 20:34:36.151248 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:36.151234    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf5e104a-6fca-47cf-a949-5fb3f21e142e" path="/var/lib/kubelet/pods/cf5e104a-6fca-47cf-a949-5fb3f21e142e/volumes"
Sep 15 20:34:36.868940 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:36.868868    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:37.530782 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:37.530728    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:38.367222 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:38.367124    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:39.057902 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:39.057858    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:39.846440 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:39.846406    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:40.478899 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:40.478857    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:41.702310 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:41.702273    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:41.803277 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:41.803220    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:41.993812 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:41.993777    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:42.419088 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:42.419016    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-dc12d7-nbrdx"]
Sep 15 20:34:42.419491 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:42.419466    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c456c132-4bb9-4325-a972-ccc0057ef664" containerName="payload-processing"
Sep 15 20:34:42.419541 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:42.419493    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c456c132-4bb9-4325-a972-ccc0057ef664" containerName="payload-processing"
Sep 15 20:34:42.419541 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:42.419508    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf5e104a-6fca-47cf-a949-5fb3f21e142e" containerName="revoke-keys"
Sep 15 20:34:42.419541 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:42.419514    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf5e104a-6fca-47cf-a949-5fb3f21e142e" containerName="revoke-keys"
Sep 15 20:34:42.419640 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:42.419589    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf5e104a-6fca-47cf-a949-5fb3f21e142e" containerName="revoke-keys"
Sep 15 20:34:42.419640 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:42.419598    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c456c132-4bb9-4325-a972-ccc0057ef664" containerName="payload-processing"
Sep 15 20:34:42.424084 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:42.424066    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-dc12d7-nbrdx"
Sep 15 20:34:42.426336 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:42.426312    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 20:34:42.426941 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:42.426894    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r5lpt\""
Sep 15 20:34:42.430066 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:42.430031    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-dc12d7-nbrdx"]
Sep 15 20:34:42.467754 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:42.467731    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:42.487879 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:42.487854    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efca3da3-ac68-4bfb-aa1c-4672b38c5894-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-dc12d7-nbrdx\" (UID: \"efca3da3-ac68-4bfb-aa1c-4672b38c5894\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-dc12d7-nbrdx"
Sep 15 20:34:42.588764 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:42.588736    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efca3da3-ac68-4bfb-aa1c-4672b38c5894-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-dc12d7-nbrdx\" (UID: \"efca3da3-ac68-4bfb-aa1c-4672b38c5894\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-dc12d7-nbrdx"
Sep 15 20:34:42.589433 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:42.589413    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efca3da3-ac68-4bfb-aa1c-4672b38c5894-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-dc12d7-nbrdx\" (UID: \"efca3da3-ac68-4bfb-aa1c-4672b38c5894\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-dc12d7-nbrdx"
Sep 15 20:34:42.739138 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:42.739096    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-dc12d7-nbrdx"
Sep 15 20:34:42.863242 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:42.863209    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:43.009698 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:43.009672    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-dc12d7-nbrdx"]
Sep 15 20:34:43.011221 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:34:43.011191    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefca3da3_ac68_4bfb_aa1c_4672b38c5894.slice/crio-9b4dec6e14a04c6d1b50f7e514c7a575782b947ab39431c0936a4cdc1067a646 WatchSource:0}: Error finding container 9b4dec6e14a04c6d1b50f7e514c7a575782b947ab39431c0936a4cdc1067a646: Status 404 returned error can't find the container with id 9b4dec6e14a04c6d1b50f7e514c7a575782b947ab39431c0936a4cdc1067a646
Sep 15 20:34:43.617771 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:43.617735    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-dc12d7-nbrdx" event={"ID":"efca3da3-ac68-4bfb-aa1c-4672b38c5894","Type":"ContainerStarted","Data":"efe19cdedc01f5bb9955b7c665a49dc3f53537f9622c634484d7309ba6c673b5"}
Sep 15 20:34:43.617982 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:43.617790    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-dc12d7-nbrdx" event={"ID":"efca3da3-ac68-4bfb-aa1c-4672b38c5894","Type":"ContainerStarted","Data":"9b4dec6e14a04c6d1b50f7e514c7a575782b947ab39431c0936a4cdc1067a646"}
Sep 15 20:34:43.633495 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:43.633448    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-dc12d7-nbrdx" podStartSLOduration=1.633433237 podStartE2EDuration="1.633433237s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:34:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:34:43.631140386 +0000 UTC m=+1798.119604871" watchObservedRunningTime="2026-09-15 20:34:43.633433237 +0000 UTC m=+1798.121897713"
Sep 15 20:34:43.672615 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:43.672584    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:44.297457 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:44.297426    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:44.626235 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:44.626137    2576 generic.go:356] "Generic (PLEG): container finished" podID="efca3da3-ac68-4bfb-aa1c-4672b38c5894" containerID="efe19cdedc01f5bb9955b7c665a49dc3f53537f9622c634484d7309ba6c673b5" exitCode=0
Sep 15 20:34:44.626235 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:44.626226    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-dc12d7-nbrdx" event={"ID":"efca3da3-ac68-4bfb-aa1c-4672b38c5894","Type":"ContainerDied","Data":"efe19cdedc01f5bb9955b7c665a49dc3f53537f9622c634484d7309ba6c673b5"}
Sep 15 20:34:45.160750 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:45.160719    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:45.776386 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:45.776352    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:45.778721 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:45.778701    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-dc12d7-nbrdx"
Sep 15 20:34:45.922587 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:45.922508    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efca3da3-ac68-4bfb-aa1c-4672b38c5894-maas-api-service-ca\") pod \"efca3da3-ac68-4bfb-aa1c-4672b38c5894\" (UID: \"efca3da3-ac68-4bfb-aa1c-4672b38c5894\") "
Sep 15 20:34:45.922872 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:45.922850    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efca3da3-ac68-4bfb-aa1c-4672b38c5894-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "efca3da3-ac68-4bfb-aa1c-4672b38c5894" (UID: "efca3da3-ac68-4bfb-aa1c-4672b38c5894"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:34:46.023656 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:46.023624    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efca3da3-ac68-4bfb-aa1c-4672b38c5894-maas-api-service-ca\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:34:46.565523 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:46.565424    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:46.638359 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:46.638332    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-dc12d7-nbrdx"
Sep 15 20:34:46.638359 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:46.638350    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-dc12d7-nbrdx" event={"ID":"efca3da3-ac68-4bfb-aa1c-4672b38c5894","Type":"ContainerDied","Data":"9b4dec6e14a04c6d1b50f7e514c7a575782b947ab39431c0936a4cdc1067a646"}
Sep 15 20:34:46.638599 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:46.638401    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9b4dec6e14a04c6d1b50f7e514c7a575782b947ab39431c0936a4cdc1067a646"
Sep 15 20:34:47.258917 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:47.258884    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:47.456740 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:47.456694    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh"]
Sep 15 20:34:47.497732 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:47.497698    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt"]
Sep 15 20:34:47.601296 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:47.601219    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:48.227759 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:48.227729    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:48.972675 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:48.972642    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:52.674085 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:52.674053    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:53.344363 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:53.344330    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-dc12d7-29825070-2k4xs"]
Sep 15 20:34:53.346532 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:53.346512    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-dc12d7-29825070-2k4xs"]
Sep 15 20:34:53.523887 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:53.523846    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-dc12d7-nbrdx"]
Sep 15 20:34:53.526198 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:53.526175    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-dc12d7-nbrdx"]
Sep 15 20:34:53.819662 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:53.819628    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:53.919884 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:53.919859    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:54.021444 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:54.021415    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:54.144201 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:54.144128    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fbab977-092b-41df-ab62-0542e81e4571" path="/var/lib/kubelet/pods/3fbab977-092b-41df-ab62-0542e81e4571/volumes"
Sep 15 20:34:54.144507 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:54.144492    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efca3da3-ac68-4bfb-aa1c-4672b38c5894" path="/var/lib/kubelet/pods/efca3da3-ac68-4bfb-aa1c-4672b38c5894/volumes"
Sep 15 20:34:54.777353 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:54.777316    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:55.461882 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:55.461839    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:56.371379 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:56.371346    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:56.985463 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:56.985420    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:57.602206 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:57.602168    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:57.602974 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:57.602882    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7e3543-29825070-mkmrz"]
Sep 15 20:34:57.605127 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:57.605106    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7e3543-29825070-mkmrz"]
Sep 15 20:34:57.624775 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:57.624749    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-7e3543-59dfc588fc-knvf7"]
Sep 15 20:34:57.625049 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:57.625014    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-7e3543-59dfc588fc-knvf7" podUID="a7912229-70e1-4ec7-8c05-7f756a8a9411" containerName="payload-processing" containerID="cri-o://ba541365fec731519309143327d4f1fdddc9e8b495f5c18ab18adfc555882851" gracePeriod=30
Sep 15 20:34:57.632969 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:57.632944    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-7e3543-649dcfdd74-nm9vg"]
Sep 15 20:34:57.633613 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:57.633260    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7e3543-649dcfdd74-nm9vg" podUID="95418b12-f771-4e7c-836e-bb5e04eac166" containerName="payload-pre-processing" containerID="cri-o://075039eefb0f02375bfa33c7c5cb9bcb15698e7609a646a41a089c82b12d21e9" gracePeriod=30
Sep 15 20:34:57.677446 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:57.677418    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:57.814223 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:57.814192    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:57.881010 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:57.880970    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:57.907519 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:57.907500    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-7e3543-59dfc588fc-knvf7"
Sep 15 20:34:57.910676 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:57.910654    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7e3543-649dcfdd74-nm9vg"
Sep 15 20:34:57.948069 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:57.948042    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95418b12-f771-4e7c-836e-bb5e04eac166-plugins-config-volume\") pod \"95418b12-f771-4e7c-836e-bb5e04eac166\" (UID: \"95418b12-f771-4e7c-836e-bb5e04eac166\") "
Sep 15 20:34:57.948218 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:57.948107    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5rnl\" (UniqueName: \"kubernetes.io/projected/95418b12-f771-4e7c-836e-bb5e04eac166-kube-api-access-s5rnl\") pod \"95418b12-f771-4e7c-836e-bb5e04eac166\" (UID: \"95418b12-f771-4e7c-836e-bb5e04eac166\") "
Sep 15 20:34:57.948295 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:57.948223    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7912229-70e1-4ec7-8c05-7f756a8a9411-plugins-config-volume\") pod \"a7912229-70e1-4ec7-8c05-7f756a8a9411\" (UID: \"a7912229-70e1-4ec7-8c05-7f756a8a9411\") "
Sep 15 20:34:57.948295 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:57.948261    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kcpw2\" (UniqueName: \"kubernetes.io/projected/a7912229-70e1-4ec7-8c05-7f756a8a9411-kube-api-access-kcpw2\") pod \"a7912229-70e1-4ec7-8c05-7f756a8a9411\" (UID: \"a7912229-70e1-4ec7-8c05-7f756a8a9411\") "
Sep 15 20:34:57.948448 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:57.948418    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95418b12-f771-4e7c-836e-bb5e04eac166-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "95418b12-f771-4e7c-836e-bb5e04eac166" (UID: "95418b12-f771-4e7c-836e-bb5e04eac166"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:34:57.948599 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:57.948581    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95418b12-f771-4e7c-836e-bb5e04eac166-plugins-config-volume\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:34:57.949058 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:57.948996    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7912229-70e1-4ec7-8c05-7f756a8a9411-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a7912229-70e1-4ec7-8c05-7f756a8a9411" (UID: "a7912229-70e1-4ec7-8c05-7f756a8a9411"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:34:57.949348 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:57.949325    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:57.950700 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:57.950670    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95418b12-f771-4e7c-836e-bb5e04eac166-kube-api-access-s5rnl" (OuterVolumeSpecName: "kube-api-access-s5rnl") pod "95418b12-f771-4e7c-836e-bb5e04eac166" (UID: "95418b12-f771-4e7c-836e-bb5e04eac166"). InnerVolumeSpecName "kube-api-access-s5rnl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:34:57.950979 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:57.950959    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7912229-70e1-4ec7-8c05-7f756a8a9411-kube-api-access-kcpw2" (OuterVolumeSpecName: "kube-api-access-kcpw2") pod "a7912229-70e1-4ec7-8c05-7f756a8a9411" (UID: "a7912229-70e1-4ec7-8c05-7f756a8a9411"). InnerVolumeSpecName "kube-api-access-kcpw2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:34:58.049558 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:58.049518    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5rnl\" (UniqueName: \"kubernetes.io/projected/95418b12-f771-4e7c-836e-bb5e04eac166-kube-api-access-s5rnl\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:34:58.049558 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:58.049553    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7912229-70e1-4ec7-8c05-7f756a8a9411-plugins-config-volume\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:34:58.049558 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:58.049564    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kcpw2\" (UniqueName: \"kubernetes.io/projected/a7912229-70e1-4ec7-8c05-7f756a8a9411-kube-api-access-kcpw2\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:34:58.143902 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:58.143830    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11bd406d-e791-4bb2-950b-93b944a22327" path="/var/lib/kubelet/pods/11bd406d-e791-4bb2-950b-93b944a22327/volumes"
Sep 15 20:34:58.338976 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:58.338943    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:58.439677 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:58.439596    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:58.707231 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:58.707192    2576 generic.go:356] "Generic (PLEG): container finished" podID="95418b12-f771-4e7c-836e-bb5e04eac166" containerID="075039eefb0f02375bfa33c7c5cb9bcb15698e7609a646a41a089c82b12d21e9" exitCode=0
Sep 15 20:34:58.707392 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:58.707233    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7e3543-649dcfdd74-nm9vg"
Sep 15 20:34:58.707392 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:58.707273    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7e3543-649dcfdd74-nm9vg" event={"ID":"95418b12-f771-4e7c-836e-bb5e04eac166","Type":"ContainerDied","Data":"075039eefb0f02375bfa33c7c5cb9bcb15698e7609a646a41a089c82b12d21e9"}
Sep 15 20:34:58.707392 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:58.707324    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7e3543-649dcfdd74-nm9vg" event={"ID":"95418b12-f771-4e7c-836e-bb5e04eac166","Type":"ContainerDied","Data":"411d05c9814b3525b9646c66129efc77fed110febc97c07a1c2faa075720012d"}
Sep 15 20:34:58.707392 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:58.707347    2576 scope.go:117] "RemoveContainer" containerID="075039eefb0f02375bfa33c7c5cb9bcb15698e7609a646a41a089c82b12d21e9"
Sep 15 20:34:58.708903 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:58.708881    2576 generic.go:356] "Generic (PLEG): container finished" podID="a7912229-70e1-4ec7-8c05-7f756a8a9411" containerID="ba541365fec731519309143327d4f1fdddc9e8b495f5c18ab18adfc555882851" exitCode=0
Sep 15 20:34:58.709078 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:58.708916    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-7e3543-59dfc588fc-knvf7"
Sep 15 20:34:58.709078 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:58.708953    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-7e3543-59dfc588fc-knvf7" event={"ID":"a7912229-70e1-4ec7-8c05-7f756a8a9411","Type":"ContainerDied","Data":"ba541365fec731519309143327d4f1fdddc9e8b495f5c18ab18adfc555882851"}
Sep 15 20:34:58.709078 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:58.708989    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-7e3543-59dfc588fc-knvf7" event={"ID":"a7912229-70e1-4ec7-8c05-7f756a8a9411","Type":"ContainerDied","Data":"cad2d2bb330d278ed248aa56e17d5cc12428dfb021649aff67ca8d26736863b4"}
Sep 15 20:34:58.716438 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:58.716418    2576 scope.go:117] "RemoveContainer" containerID="075039eefb0f02375bfa33c7c5cb9bcb15698e7609a646a41a089c82b12d21e9"
Sep 15 20:34:58.716691 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:34:58.716672    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"075039eefb0f02375bfa33c7c5cb9bcb15698e7609a646a41a089c82b12d21e9\": container with ID starting with 075039eefb0f02375bfa33c7c5cb9bcb15698e7609a646a41a089c82b12d21e9 not found: ID does not exist" containerID="075039eefb0f02375bfa33c7c5cb9bcb15698e7609a646a41a089c82b12d21e9"
Sep 15 20:34:58.716754 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:58.716699    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"075039eefb0f02375bfa33c7c5cb9bcb15698e7609a646a41a089c82b12d21e9"} err="failed to get container status \"075039eefb0f02375bfa33c7c5cb9bcb15698e7609a646a41a089c82b12d21e9\": rpc error: code = NotFound desc = could not find container \"075039eefb0f02375bfa33c7c5cb9bcb15698e7609a646a41a089c82b12d21e9\": container with ID starting with 075039eefb0f02375bfa33c7c5cb9bcb15698e7609a646a41a089c82b12d21e9 not found: ID does not exist"
Sep 15 20:34:58.716754 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:58.716714    2576 scope.go:117] "RemoveContainer" containerID="ba541365fec731519309143327d4f1fdddc9e8b495f5c18ab18adfc555882851"
Sep 15 20:34:58.724821 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:58.724722    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-7e3543-649dcfdd74-nm9vg"]
Sep 15 20:34:58.724870 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:58.724847    2576 scope.go:117] "RemoveContainer" containerID="ba541365fec731519309143327d4f1fdddc9e8b495f5c18ab18adfc555882851"
Sep 15 20:34:58.725138 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:34:58.725121    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba541365fec731519309143327d4f1fdddc9e8b495f5c18ab18adfc555882851\": container with ID starting with ba541365fec731519309143327d4f1fdddc9e8b495f5c18ab18adfc555882851 not found: ID does not exist" containerID="ba541365fec731519309143327d4f1fdddc9e8b495f5c18ab18adfc555882851"
Sep 15 20:34:58.725197 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:58.725145    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba541365fec731519309143327d4f1fdddc9e8b495f5c18ab18adfc555882851"} err="failed to get container status \"ba541365fec731519309143327d4f1fdddc9e8b495f5c18ab18adfc555882851\": rpc error: code = NotFound desc = could not find container \"ba541365fec731519309143327d4f1fdddc9e8b495f5c18ab18adfc555882851\": container with ID starting with ba541365fec731519309143327d4f1fdddc9e8b495f5c18ab18adfc555882851 not found: ID does not exist"
Sep 15 20:34:58.731724 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:58.731704    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-7e3543-649dcfdd74-nm9vg"]
Sep 15 20:34:58.742377 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:58.742354    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-7e3543-59dfc588fc-knvf7"]
Sep 15 20:34:58.745652 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:58.745631    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-7e3543-59dfc588fc-knvf7"]
Sep 15 20:34:58.994548 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:58.994466    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:34:59.791973 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:34:59.791903    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:35:00.145550 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:00.145462    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95418b12-f771-4e7c-836e-bb5e04eac166" path="/var/lib/kubelet/pods/95418b12-f771-4e7c-836e-bb5e04eac166/volumes"
Sep 15 20:35:00.145874 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:00.145858    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7912229-70e1-4ec7-8c05-7f756a8a9411" path="/var/lib/kubelet/pods/a7912229-70e1-4ec7-8c05-7f756a8a9411/volumes"
Sep 15 20:35:00.436592 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:00.436483    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:35:00.539249 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:00.539206    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:35:00.770790 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:00.770757    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:35:01.349905 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:01.349864    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:35:02.435143 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.435062    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:35:02.477132 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.477089    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh" podUID="2cd2c1ab-b253-40fd-b902-5de5943ab450" containerName="main" containerID="cri-o://63ebcb390653f52a02e5329c8b180fc25f0d5fb9214b3b64a1fb8b89ef2c0900" gracePeriod=45
Sep 15 20:35:02.516592 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.516543    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt" podUID="68aa3f0c-93b1-44e7-b21e-3e5353baea2a" containerName="main" containerID="cri-o://77668b2a36e91db94d52087a08d1b465ade209ebbad05b97c5105328b727b7fc" gracePeriod=45
Sep 15 20:35:02.738260 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.738229    2576 generic.go:356] "Generic (PLEG): container finished" podID="2cd2c1ab-b253-40fd-b902-5de5943ab450" containerID="63ebcb390653f52a02e5329c8b180fc25f0d5fb9214b3b64a1fb8b89ef2c0900" exitCode=0
Sep 15 20:35:02.738380 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.738300    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh" event={"ID":"2cd2c1ab-b253-40fd-b902-5de5943ab450","Type":"ContainerDied","Data":"63ebcb390653f52a02e5329c8b180fc25f0d5fb9214b3b64a1fb8b89ef2c0900"}
Sep 15 20:35:02.740441 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.740412    2576 generic.go:356] "Generic (PLEG): container finished" podID="68aa3f0c-93b1-44e7-b21e-3e5353baea2a" containerID="77668b2a36e91db94d52087a08d1b465ade209ebbad05b97c5105328b727b7fc" exitCode=0
Sep 15 20:35:02.740557 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.740498    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt" event={"ID":"68aa3f0c-93b1-44e7-b21e-3e5353baea2a","Type":"ContainerDied","Data":"77668b2a36e91db94d52087a08d1b465ade209ebbad05b97c5105328b727b7fc"}
Sep 15 20:35:02.802388 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.802120    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh"
Sep 15 20:35:02.831661 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.831631    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt"
Sep 15 20:35:02.899697 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.899664    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpkbc\" (UniqueName: \"kubernetes.io/projected/2cd2c1ab-b253-40fd-b902-5de5943ab450-kube-api-access-cpkbc\") pod \"2cd2c1ab-b253-40fd-b902-5de5943ab450\" (UID: \"2cd2c1ab-b253-40fd-b902-5de5943ab450\") "
Sep 15 20:35:02.899870 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.899728    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2cd2c1ab-b253-40fd-b902-5de5943ab450-home\") pod \"2cd2c1ab-b253-40fd-b902-5de5943ab450\" (UID: \"2cd2c1ab-b253-40fd-b902-5de5943ab450\") "
Sep 15 20:35:02.899870 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.899746    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2cd2c1ab-b253-40fd-b902-5de5943ab450-tmp-dir\") pod \"2cd2c1ab-b253-40fd-b902-5de5943ab450\" (UID: \"2cd2c1ab-b253-40fd-b902-5de5943ab450\") "
Sep 15 20:35:02.899870 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.899766    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-tls-certs\") pod \"68aa3f0c-93b1-44e7-b21e-3e5353baea2a\" (UID: \"68aa3f0c-93b1-44e7-b21e-3e5353baea2a\") "
Sep 15 20:35:02.899870 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.899796    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2cd2c1ab-b253-40fd-b902-5de5943ab450-tls-certs\") pod \"2cd2c1ab-b253-40fd-b902-5de5943ab450\" (UID: \"2cd2c1ab-b253-40fd-b902-5de5943ab450\") "
Sep 15 20:35:02.899870 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.899816    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-tmp-dir\") pod \"68aa3f0c-93b1-44e7-b21e-3e5353baea2a\" (UID: \"68aa3f0c-93b1-44e7-b21e-3e5353baea2a\") "
Sep 15 20:35:02.900165 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.899873    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-home\") pod \"68aa3f0c-93b1-44e7-b21e-3e5353baea2a\" (UID: \"68aa3f0c-93b1-44e7-b21e-3e5353baea2a\") "
Sep 15 20:35:02.900165 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.899902    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-model-cache\") pod \"68aa3f0c-93b1-44e7-b21e-3e5353baea2a\" (UID: \"68aa3f0c-93b1-44e7-b21e-3e5353baea2a\") "
Sep 15 20:35:02.900165 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.899971    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2cd2c1ab-b253-40fd-b902-5de5943ab450-model-cache\") pod \"2cd2c1ab-b253-40fd-b902-5de5943ab450\" (UID: \"2cd2c1ab-b253-40fd-b902-5de5943ab450\") "
Sep 15 20:35:02.900165 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.900005    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2cd2c1ab-b253-40fd-b902-5de5943ab450-dshm\") pod \"2cd2c1ab-b253-40fd-b902-5de5943ab450\" (UID: \"2cd2c1ab-b253-40fd-b902-5de5943ab450\") "
Sep 15 20:35:02.900165 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.900045    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-dshm\") pod \"68aa3f0c-93b1-44e7-b21e-3e5353baea2a\" (UID: \"68aa3f0c-93b1-44e7-b21e-3e5353baea2a\") "
Sep 15 20:35:02.900165 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.900072    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96w7f\" (UniqueName: \"kubernetes.io/projected/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-kube-api-access-96w7f\") pod \"68aa3f0c-93b1-44e7-b21e-3e5353baea2a\" (UID: \"68aa3f0c-93b1-44e7-b21e-3e5353baea2a\") "
Sep 15 20:35:02.900165 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.900089    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cd2c1ab-b253-40fd-b902-5de5943ab450-home" (OuterVolumeSpecName: "home") pod "2cd2c1ab-b253-40fd-b902-5de5943ab450" (UID: "2cd2c1ab-b253-40fd-b902-5de5943ab450"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:35:02.900165 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.900105    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cd2c1ab-b253-40fd-b902-5de5943ab450-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2cd2c1ab-b253-40fd-b902-5de5943ab450" (UID: "2cd2c1ab-b253-40fd-b902-5de5943ab450"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:35:02.900583 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.900455    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2cd2c1ab-b253-40fd-b902-5de5943ab450-home\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:35:02.900583 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.900478    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2cd2c1ab-b253-40fd-b902-5de5943ab450-tmp-dir\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:35:02.900583 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.900518    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cd2c1ab-b253-40fd-b902-5de5943ab450-model-cache" (OuterVolumeSpecName: "model-cache") pod "2cd2c1ab-b253-40fd-b902-5de5943ab450" (UID: "2cd2c1ab-b253-40fd-b902-5de5943ab450"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:35:02.901131 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.900752    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "68aa3f0c-93b1-44e7-b21e-3e5353baea2a" (UID: "68aa3f0c-93b1-44e7-b21e-3e5353baea2a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:35:02.901131 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.901016    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-home" (OuterVolumeSpecName: "home") pod "68aa3f0c-93b1-44e7-b21e-3e5353baea2a" (UID: "68aa3f0c-93b1-44e7-b21e-3e5353baea2a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:35:02.901131 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.901105    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-model-cache" (OuterVolumeSpecName: "model-cache") pod "68aa3f0c-93b1-44e7-b21e-3e5353baea2a" (UID: "68aa3f0c-93b1-44e7-b21e-3e5353baea2a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:35:02.902612 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.902576    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cd2c1ab-b253-40fd-b902-5de5943ab450-kube-api-access-cpkbc" (OuterVolumeSpecName: "kube-api-access-cpkbc") pod "2cd2c1ab-b253-40fd-b902-5de5943ab450" (UID: "2cd2c1ab-b253-40fd-b902-5de5943ab450"). InnerVolumeSpecName "kube-api-access-cpkbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:35:02.902612 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.902586    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2cd2c1ab-b253-40fd-b902-5de5943ab450-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2cd2c1ab-b253-40fd-b902-5de5943ab450" (UID: "2cd2c1ab-b253-40fd-b902-5de5943ab450"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:35:02.902804 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.902716    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cd2c1ab-b253-40fd-b902-5de5943ab450-dshm" (OuterVolumeSpecName: "dshm") pod "2cd2c1ab-b253-40fd-b902-5de5943ab450" (UID: "2cd2c1ab-b253-40fd-b902-5de5943ab450"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:35:02.902804 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.902763    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "68aa3f0c-93b1-44e7-b21e-3e5353baea2a" (UID: "68aa3f0c-93b1-44e7-b21e-3e5353baea2a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:35:02.902996 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.902980    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-kube-api-access-96w7f" (OuterVolumeSpecName: "kube-api-access-96w7f") pod "68aa3f0c-93b1-44e7-b21e-3e5353baea2a" (UID: "68aa3f0c-93b1-44e7-b21e-3e5353baea2a"). InnerVolumeSpecName "kube-api-access-96w7f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:35:02.903072 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.902993    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-dshm" (OuterVolumeSpecName: "dshm") pod "68aa3f0c-93b1-44e7-b21e-3e5353baea2a" (UID: "68aa3f0c-93b1-44e7-b21e-3e5353baea2a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:35:02.970844 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:02.970807    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:35:03.001009 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:03.000977    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpkbc\" (UniqueName: \"kubernetes.io/projected/2cd2c1ab-b253-40fd-b902-5de5943ab450-kube-api-access-cpkbc\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:35:03.001009 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:03.001004    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-tls-certs\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:35:03.001009 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:03.001014    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2cd2c1ab-b253-40fd-b902-5de5943ab450-tls-certs\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:35:03.001236 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:03.001028    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-tmp-dir\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:35:03.001236 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:03.001037    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-home\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:35:03.001236 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:03.001046    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-model-cache\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:35:03.001236 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:03.001053    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2cd2c1ab-b253-40fd-b902-5de5943ab450-model-cache\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:35:03.001236 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:03.001060    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2cd2c1ab-b253-40fd-b902-5de5943ab450-dshm\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:35:03.001236 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:03.001066    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-dshm\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:35:03.001236 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:03.001074    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96w7f\" (UniqueName: \"kubernetes.io/projected/68aa3f0c-93b1-44e7-b21e-3e5353baea2a-kube-api-access-96w7f\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:35:03.591896 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:03.591863    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:35:03.748372 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:03.748290    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh"
Sep 15 20:35:03.748372 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:03.748339    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh" event={"ID":"2cd2c1ab-b253-40fd-b902-5de5943ab450","Type":"ContainerDied","Data":"7f1784cac342d4e736ccd1501ea3889f9508be5cc4cb9d78750d37b0bcd1209b"}
Sep 15 20:35:03.748603 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:03.748393    2576 scope.go:117] "RemoveContainer" containerID="63ebcb390653f52a02e5329c8b180fc25f0d5fb9214b3b64a1fb8b89ef2c0900"
Sep 15 20:35:03.749880 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:03.749857    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt"
Sep 15 20:35:03.749880 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:03.749868    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt" event={"ID":"68aa3f0c-93b1-44e7-b21e-3e5353baea2a","Type":"ContainerDied","Data":"77f527e095777ae1324a8b871bbc9b3ad38a9a93475397b9a101453e7934ef86"}
Sep 15 20:35:03.757515 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:03.757498    2576 scope.go:117] "RemoveContainer" containerID="77668b2a36e91db94d52087a08d1b465ade209ebbad05b97c5105328b727b7fc"
Sep 15 20:35:03.773762 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:03.773734    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt"]
Sep 15 20:35:03.776929 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:03.776903    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-7e3543/simulator-w0-7e3543-kserve-5fb6d4d6cd-pvxwt"]
Sep 15 20:35:03.786549 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:03.786525    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh"]
Sep 15 20:35:03.788653 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:03.788636    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-7e3543/distinct-2-w0-7e3543-kserve-6c458dc8d9-vzzqh"]
Sep 15 20:35:04.143845 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:04.143766    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2cd2c1ab-b253-40fd-b902-5de5943ab450" path="/var/lib/kubelet/pods/2cd2c1ab-b253-40fd-b902-5de5943ab450/volumes"
Sep 15 20:35:04.144212 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:04.144198    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68aa3f0c-93b1-44e7-b21e-3e5353baea2a" path="/var/lib/kubelet/pods/68aa3f0c-93b1-44e7-b21e-3e5353baea2a/volumes"
Sep 15 20:35:04.253462 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:04.253424    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:35:04.695604 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:04.695574    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:35:04.796077 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:04.796042    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:35:04.900542 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:04.900508    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:35:05.003635 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:05.003605    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:35:11.347429 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:11.347395    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:35:11.873114 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:11.873085    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:35:11.879449 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:11.879421    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f49968-29825070-qh7l4"]
Sep 15 20:35:11.881864 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:11.881843    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f49968-29825070-qh7l4"]
Sep 15 20:35:11.910103 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:11.909935    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f49968-8658b75798-b4dxt"]
Sep 15 20:35:11.910864 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:11.910447    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f49968-8658b75798-b4dxt" podUID="76689ad7-ea6b-44cf-bb1b-2190a608e81e" containerName="payload-pre-processing" containerID="cri-o://c8e6a2050a38a06a01563fa44eb4cece3eecb303102519ae6982dedf2cf06cb2" gracePeriod=30
Sep 15 20:35:11.937122 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:11.937099    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:35:12.000272 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:12.000247    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:35:12.058668 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:12.058125    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:35:12.114842 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:12.114814    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:35:12.146834 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:12.146749    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7302c55-4f1d-4aa3-80a7-0742f0435ec6" path="/var/lib/kubelet/pods/f7302c55-4f1d-4aa3-80a7-0742f0435ec6/volumes"
Sep 15 20:35:12.182104 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:12.182079    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f49968-8658b75798-b4dxt"
Sep 15 20:35:12.286203 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:12.286178    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwr2z\" (UniqueName: \"kubernetes.io/projected/76689ad7-ea6b-44cf-bb1b-2190a608e81e-kube-api-access-vwr2z\") pod \"76689ad7-ea6b-44cf-bb1b-2190a608e81e\" (UID: \"76689ad7-ea6b-44cf-bb1b-2190a608e81e\") "
Sep 15 20:35:12.286341 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:12.286280    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76689ad7-ea6b-44cf-bb1b-2190a608e81e-plugins-config-volume\") pod \"76689ad7-ea6b-44cf-bb1b-2190a608e81e\" (UID: \"76689ad7-ea6b-44cf-bb1b-2190a608e81e\") "
Sep 15 20:35:12.286619 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:12.286594    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76689ad7-ea6b-44cf-bb1b-2190a608e81e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "76689ad7-ea6b-44cf-bb1b-2190a608e81e" (UID: "76689ad7-ea6b-44cf-bb1b-2190a608e81e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:35:12.288001 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:12.287981    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76689ad7-ea6b-44cf-bb1b-2190a608e81e-kube-api-access-vwr2z" (OuterVolumeSpecName: "kube-api-access-vwr2z") pod "76689ad7-ea6b-44cf-bb1b-2190a608e81e" (UID: "76689ad7-ea6b-44cf-bb1b-2190a608e81e"). InnerVolumeSpecName "kube-api-access-vwr2z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:35:12.387033 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:12.387007    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwr2z\" (UniqueName: \"kubernetes.io/projected/76689ad7-ea6b-44cf-bb1b-2190a608e81e-kube-api-access-vwr2z\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:35:12.387033 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:12.387028    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76689ad7-ea6b-44cf-bb1b-2190a608e81e-plugins-config-volume\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:35:12.468058 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:12.468032    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:35:12.577103 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:12.577076    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:35:12.801805 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:12.801731    2576 generic.go:356] "Generic (PLEG): container finished" podID="76689ad7-ea6b-44cf-bb1b-2190a608e81e" containerID="c8e6a2050a38a06a01563fa44eb4cece3eecb303102519ae6982dedf2cf06cb2" exitCode=0
Sep 15 20:35:12.801805 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:12.801775    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f49968-8658b75798-b4dxt"
Sep 15 20:35:12.801805 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:12.801783    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f49968-8658b75798-b4dxt" event={"ID":"76689ad7-ea6b-44cf-bb1b-2190a608e81e","Type":"ContainerDied","Data":"c8e6a2050a38a06a01563fa44eb4cece3eecb303102519ae6982dedf2cf06cb2"}
Sep 15 20:35:12.802048 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:12.801824    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f49968-8658b75798-b4dxt" event={"ID":"76689ad7-ea6b-44cf-bb1b-2190a608e81e","Type":"ContainerDied","Data":"2cee0f08bc0e9caf5d057ed6b9e8c272ddb46a6ca1e85b3895f069e989ba3837"}
Sep 15 20:35:12.802048 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:12.801853    2576 scope.go:117] "RemoveContainer" containerID="c8e6a2050a38a06a01563fa44eb4cece3eecb303102519ae6982dedf2cf06cb2"
Sep 15 20:35:12.811303 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:12.811287    2576 scope.go:117] "RemoveContainer" containerID="c8e6a2050a38a06a01563fa44eb4cece3eecb303102519ae6982dedf2cf06cb2"
Sep 15 20:35:12.811593 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:35:12.811574    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8e6a2050a38a06a01563fa44eb4cece3eecb303102519ae6982dedf2cf06cb2\": container with ID starting with c8e6a2050a38a06a01563fa44eb4cece3eecb303102519ae6982dedf2cf06cb2 not found: ID does not exist" containerID="c8e6a2050a38a06a01563fa44eb4cece3eecb303102519ae6982dedf2cf06cb2"
Sep 15 20:35:12.811647 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:12.811601    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8e6a2050a38a06a01563fa44eb4cece3eecb303102519ae6982dedf2cf06cb2"} err="failed to get container status \"c8e6a2050a38a06a01563fa44eb4cece3eecb303102519ae6982dedf2cf06cb2\": rpc error: code = NotFound desc = could not find container \"c8e6a2050a38a06a01563fa44eb4cece3eecb303102519ae6982dedf2cf06cb2\": container with ID starting with c8e6a2050a38a06a01563fa44eb4cece3eecb303102519ae6982dedf2cf06cb2 not found: ID does not exist"
Sep 15 20:35:12.824567 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:12.824539    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f49968-8658b75798-b4dxt"]
Sep 15 20:35:12.827988 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:12.827963    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f49968-8658b75798-b4dxt"]
Sep 15 20:35:13.246670 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:13.246642    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:35:14.058553 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:14.058517    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:35:14.143298 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:14.143265    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76689ad7-ea6b-44cf-bb1b-2190a608e81e" path="/var/lib/kubelet/pods/76689ad7-ea6b-44cf-bb1b-2190a608e81e/volumes"
Sep 15 20:35:14.696469 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:14.696435    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:35:18.091222 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:18.091186    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:35:18.160932 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:18.160899    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:35:18.392963 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:18.392874    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:35:20.266839 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:20.266809    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:35:24.487112 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:24.487050    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:35:24.577468 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:24.577434    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:35:25.049590 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:25.049550    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:35:26.665501 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:26.665467    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:35:26.728636 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:26.728601    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:35:46.222774 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:46.222746    2576 scope.go:117] "RemoveContainer" containerID="9380fb6dbb38c7c4153a9f4135fe4e5cff06cc905d61f8de5a1e5774315fcd10"
Sep 15 20:35:56.883344 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:35:56.883311    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:36:00.460676 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:00.460643    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt"]
Sep 15 20:36:00.472577 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:00.472553    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:36:01.730382 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:01.730350    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:36:06.651620 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:06.651585    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:36:13.661405 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:13.661336    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:36:14.149618 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:14.149584    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:36:14.767880 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:14.767836    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:36:15.385541 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:15.385509    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:36:15.480236 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:15.480193    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt" podUID="8aa6c347-9591-490f-bc74-4ee372e2db2f" containerName="main" containerID="cri-o://d06c91f6eca55744eca755a3abc216b1a6f87c920031bb2b304186bef9434036" gracePeriod=45
Sep 15 20:36:15.730636 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:15.730614    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt"
Sep 15 20:36:15.856679 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:15.856644    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8aa6c347-9591-490f-bc74-4ee372e2db2f-model-cache\") pod \"8aa6c347-9591-490f-bc74-4ee372e2db2f\" (UID: \"8aa6c347-9591-490f-bc74-4ee372e2db2f\") "
Sep 15 20:36:15.857180 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:15.856722    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8aa6c347-9591-490f-bc74-4ee372e2db2f-home\") pod \"8aa6c347-9591-490f-bc74-4ee372e2db2f\" (UID: \"8aa6c347-9591-490f-bc74-4ee372e2db2f\") "
Sep 15 20:36:15.857180 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:15.856764    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8aa6c347-9591-490f-bc74-4ee372e2db2f-tls-certs\") pod \"8aa6c347-9591-490f-bc74-4ee372e2db2f\" (UID: \"8aa6c347-9591-490f-bc74-4ee372e2db2f\") "
Sep 15 20:36:15.857180 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:15.856798    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pcllj\" (UniqueName: \"kubernetes.io/projected/8aa6c347-9591-490f-bc74-4ee372e2db2f-kube-api-access-pcllj\") pod \"8aa6c347-9591-490f-bc74-4ee372e2db2f\" (UID: \"8aa6c347-9591-490f-bc74-4ee372e2db2f\") "
Sep 15 20:36:15.857180 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:15.856819    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8aa6c347-9591-490f-bc74-4ee372e2db2f-tmp-dir\") pod \"8aa6c347-9591-490f-bc74-4ee372e2db2f\" (UID: \"8aa6c347-9591-490f-bc74-4ee372e2db2f\") "
Sep 15 20:36:15.857180 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:15.856849    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8aa6c347-9591-490f-bc74-4ee372e2db2f-dshm\") pod \"8aa6c347-9591-490f-bc74-4ee372e2db2f\" (UID: \"8aa6c347-9591-490f-bc74-4ee372e2db2f\") "
Sep 15 20:36:15.857180 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:15.856984    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8aa6c347-9591-490f-bc74-4ee372e2db2f-home" (OuterVolumeSpecName: "home") pod "8aa6c347-9591-490f-bc74-4ee372e2db2f" (UID: "8aa6c347-9591-490f-bc74-4ee372e2db2f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:36:15.857180 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:15.857008    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8aa6c347-9591-490f-bc74-4ee372e2db2f-model-cache" (OuterVolumeSpecName: "model-cache") pod "8aa6c347-9591-490f-bc74-4ee372e2db2f" (UID: "8aa6c347-9591-490f-bc74-4ee372e2db2f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:36:15.857180 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:15.857156    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8aa6c347-9591-490f-bc74-4ee372e2db2f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8aa6c347-9591-490f-bc74-4ee372e2db2f" (UID: "8aa6c347-9591-490f-bc74-4ee372e2db2f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:36:15.857490 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:15.857257    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8aa6c347-9591-490f-bc74-4ee372e2db2f-home\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:36:15.857490 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:15.857274    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8aa6c347-9591-490f-bc74-4ee372e2db2f-tmp-dir\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:36:15.857490 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:15.857289    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8aa6c347-9591-490f-bc74-4ee372e2db2f-model-cache\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:36:15.859087 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:15.859061    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8aa6c347-9591-490f-bc74-4ee372e2db2f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8aa6c347-9591-490f-bc74-4ee372e2db2f" (UID: "8aa6c347-9591-490f-bc74-4ee372e2db2f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:36:15.859199 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:15.859079    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8aa6c347-9591-490f-bc74-4ee372e2db2f-dshm" (OuterVolumeSpecName: "dshm") pod "8aa6c347-9591-490f-bc74-4ee372e2db2f" (UID: "8aa6c347-9591-490f-bc74-4ee372e2db2f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:36:15.859199 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:15.859097    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8aa6c347-9591-490f-bc74-4ee372e2db2f-kube-api-access-pcllj" (OuterVolumeSpecName: "kube-api-access-pcllj") pod "8aa6c347-9591-490f-bc74-4ee372e2db2f" (UID: "8aa6c347-9591-490f-bc74-4ee372e2db2f"). InnerVolumeSpecName "kube-api-access-pcllj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:36:15.958479 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:15.958459    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8aa6c347-9591-490f-bc74-4ee372e2db2f-tls-certs\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:36:15.958607 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:15.958482    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pcllj\" (UniqueName: \"kubernetes.io/projected/8aa6c347-9591-490f-bc74-4ee372e2db2f-kube-api-access-pcllj\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:36:15.958607 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:15.958494    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8aa6c347-9591-490f-bc74-4ee372e2db2f-dshm\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:36:16.146270 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:16.146231    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:36:16.153479 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:16.153440    2576 generic.go:356] "Generic (PLEG): container finished" podID="8aa6c347-9591-490f-bc74-4ee372e2db2f" containerID="d06c91f6eca55744eca755a3abc216b1a6f87c920031bb2b304186bef9434036" exitCode=0
Sep 15 20:36:16.153629 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:16.153491    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt"
Sep 15 20:36:16.153629 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:16.153509    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt" event={"ID":"8aa6c347-9591-490f-bc74-4ee372e2db2f","Type":"ContainerDied","Data":"d06c91f6eca55744eca755a3abc216b1a6f87c920031bb2b304186bef9434036"}
Sep 15 20:36:16.153629 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:16.153546    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt" event={"ID":"8aa6c347-9591-490f-bc74-4ee372e2db2f","Type":"ContainerDied","Data":"82af52a420733221f5ddf790c64e9b14af184477a3e1a1b5abe693a6a2e5b166"}
Sep 15 20:36:16.153629 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:16.153569    2576 scope.go:117] "RemoveContainer" containerID="d06c91f6eca55744eca755a3abc216b1a6f87c920031bb2b304186bef9434036"
Sep 15 20:36:16.163684 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:16.163665    2576 scope.go:117] "RemoveContainer" containerID="d06c91f6eca55744eca755a3abc216b1a6f87c920031bb2b304186bef9434036"
Sep 15 20:36:16.163978 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:36:16.163958    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d06c91f6eca55744eca755a3abc216b1a6f87c920031bb2b304186bef9434036\": container with ID starting with d06c91f6eca55744eca755a3abc216b1a6f87c920031bb2b304186bef9434036 not found: ID does not exist" containerID="d06c91f6eca55744eca755a3abc216b1a6f87c920031bb2b304186bef9434036"
Sep 15 20:36:16.164044 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:16.163986    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d06c91f6eca55744eca755a3abc216b1a6f87c920031bb2b304186bef9434036"} err="failed to get container status \"d06c91f6eca55744eca755a3abc216b1a6f87c920031bb2b304186bef9434036\": rpc error: code = NotFound desc = could not find container \"d06c91f6eca55744eca755a3abc216b1a6f87c920031bb2b304186bef9434036\": container with ID starting with d06c91f6eca55744eca755a3abc216b1a6f87c920031bb2b304186bef9434036 not found: ID does not exist"
Sep 15 20:36:16.168156 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:16.168128    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt"]
Sep 15 20:36:16.173027 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:16.173005    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-92702a/auth-test-model-92702a-kserve-8c4d945ff-lm2dt"]
Sep 15 20:36:16.787272 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:16.787238    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:36:18.144168 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:18.144137    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8aa6c347-9591-490f-bc74-4ee372e2db2f" path="/var/lib/kubelet/pods/8aa6c347-9591-490f-bc74-4ee372e2db2f/volumes"
Sep 15 20:36:18.645696 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:18.645653    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr"]
Sep 15 20:36:18.650620 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:18.650593    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf"]
Sep 15 20:36:18.662248 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:18.662221    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8"]
Sep 15 20:36:18.729033 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:18.729001    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:36:27.723845 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:27.723810    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:36:27.773375 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:27.773340    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:36:29.377152 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:29.377119    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:36:29.387408 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:29.387381    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-725e67-29825070-jlx6q"]
Sep 15 20:36:29.389978 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:29.389954    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-725e67-29825070-jlx6q"]
Sep 15 20:36:29.410852 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:29.410824    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-725e67-667c64b48c-kz47q"]
Sep 15 20:36:29.411069 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:29.411028    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-725e67-667c64b48c-kz47q" podUID="fe36ea96-bfb6-4fd1-a260-a27872453328" containerName="payload-processing" containerID="cri-o://b6ca48352debbd5b57ef5427bb047c0091fb29d9c150c4bd232265c1ae11f3e2" gracePeriod=30
Sep 15 20:36:29.417373 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:29.417346    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-725e67-667855bfcd-cgzbp"]
Sep 15 20:36:29.417618 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:29.417570    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-725e67-667855bfcd-cgzbp" podUID="53826b73-d03f-4af1-a2aa-33825571d570" containerName="payload-pre-processing" containerID="cri-o://694e6158c90dd5d733fdf973f79b35923c49673644ee32e88b5a78e2d18adeb8" gracePeriod=30
Sep 15 20:36:29.448818 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:29.448792    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:36:29.506561 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:29.506533    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:36:29.573074 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:29.573036    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:36:29.634967 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:29.634876    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:36:29.676299 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:29.676276    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-725e67-667c64b48c-kz47q"
Sep 15 20:36:29.698572 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:29.698549    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-725e67-667855bfcd-cgzbp"
Sep 15 20:36:29.785601 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:29.785574    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe36ea96-bfb6-4fd1-a260-a27872453328-plugins-config-volume\") pod \"fe36ea96-bfb6-4fd1-a260-a27872453328\" (UID: \"fe36ea96-bfb6-4fd1-a260-a27872453328\") "
Sep 15 20:36:29.785770 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:29.785634    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wv5bn\" (UniqueName: \"kubernetes.io/projected/fe36ea96-bfb6-4fd1-a260-a27872453328-kube-api-access-wv5bn\") pod \"fe36ea96-bfb6-4fd1-a260-a27872453328\" (UID: \"fe36ea96-bfb6-4fd1-a260-a27872453328\") "
Sep 15 20:36:29.785951 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:29.785902    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe36ea96-bfb6-4fd1-a260-a27872453328-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fe36ea96-bfb6-4fd1-a260-a27872453328" (UID: "fe36ea96-bfb6-4fd1-a260-a27872453328"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:36:29.787717 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:29.787694    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe36ea96-bfb6-4fd1-a260-a27872453328-kube-api-access-wv5bn" (OuterVolumeSpecName: "kube-api-access-wv5bn") pod "fe36ea96-bfb6-4fd1-a260-a27872453328" (UID: "fe36ea96-bfb6-4fd1-a260-a27872453328"). InnerVolumeSpecName "kube-api-access-wv5bn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:36:29.886289 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:29.886220    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53826b73-d03f-4af1-a2aa-33825571d570-plugins-config-volume\") pod \"53826b73-d03f-4af1-a2aa-33825571d570\" (UID: \"53826b73-d03f-4af1-a2aa-33825571d570\") "
Sep 15 20:36:29.886289 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:29.886293    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xw722\" (UniqueName: \"kubernetes.io/projected/53826b73-d03f-4af1-a2aa-33825571d570-kube-api-access-xw722\") pod \"53826b73-d03f-4af1-a2aa-33825571d570\" (UID: \"53826b73-d03f-4af1-a2aa-33825571d570\") "
Sep 15 20:36:29.886483 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:29.886471    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wv5bn\" (UniqueName: \"kubernetes.io/projected/fe36ea96-bfb6-4fd1-a260-a27872453328-kube-api-access-wv5bn\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:36:29.886530 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:29.886485    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe36ea96-bfb6-4fd1-a260-a27872453328-plugins-config-volume\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:36:29.886565 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:29.886542    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53826b73-d03f-4af1-a2aa-33825571d570-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "53826b73-d03f-4af1-a2aa-33825571d570" (UID: "53826b73-d03f-4af1-a2aa-33825571d570"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:36:29.888259 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:29.888240    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53826b73-d03f-4af1-a2aa-33825571d570-kube-api-access-xw722" (OuterVolumeSpecName: "kube-api-access-xw722") pod "53826b73-d03f-4af1-a2aa-33825571d570" (UID: "53826b73-d03f-4af1-a2aa-33825571d570"). InnerVolumeSpecName "kube-api-access-xw722". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:36:29.965302 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:29.965279    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:36:29.987553 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:29.987522    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xw722\" (UniqueName: \"kubernetes.io/projected/53826b73-d03f-4af1-a2aa-33825571d570-kube-api-access-xw722\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:36:29.987553 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:29.987552    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53826b73-d03f-4af1-a2aa-33825571d570-plugins-config-volume\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:36:30.071414 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:30.071385    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:36:30.143004 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:30.142911    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02237da6-c778-40ec-84bf-324dea6667a0" path="/var/lib/kubelet/pods/02237da6-c778-40ec-84bf-324dea6667a0/volumes"
Sep 15 20:36:30.229931 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:30.229897    2576 generic.go:356] "Generic (PLEG): container finished" podID="53826b73-d03f-4af1-a2aa-33825571d570" containerID="694e6158c90dd5d733fdf973f79b35923c49673644ee32e88b5a78e2d18adeb8" exitCode=0
Sep 15 20:36:30.230093 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:30.229959    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-725e67-667855bfcd-cgzbp"
Sep 15 20:36:30.230093 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:30.229984    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-725e67-667855bfcd-cgzbp" event={"ID":"53826b73-d03f-4af1-a2aa-33825571d570","Type":"ContainerDied","Data":"694e6158c90dd5d733fdf973f79b35923c49673644ee32e88b5a78e2d18adeb8"}
Sep 15 20:36:30.230093 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:30.230038    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-725e67-667855bfcd-cgzbp" event={"ID":"53826b73-d03f-4af1-a2aa-33825571d570","Type":"ContainerDied","Data":"845b51d7582392f0b6467029e2dc8dce278fed8d707bed086ca2ee6c8d4a3641"}
Sep 15 20:36:30.230093 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:30.230071    2576 scope.go:117] "RemoveContainer" containerID="694e6158c90dd5d733fdf973f79b35923c49673644ee32e88b5a78e2d18adeb8"
Sep 15 20:36:30.231483 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:30.231453    2576 generic.go:356] "Generic (PLEG): container finished" podID="fe36ea96-bfb6-4fd1-a260-a27872453328" containerID="b6ca48352debbd5b57ef5427bb047c0091fb29d9c150c4bd232265c1ae11f3e2" exitCode=0
Sep 15 20:36:30.231606 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:30.231494    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-725e67-667c64b48c-kz47q" event={"ID":"fe36ea96-bfb6-4fd1-a260-a27872453328","Type":"ContainerDied","Data":"b6ca48352debbd5b57ef5427bb047c0091fb29d9c150c4bd232265c1ae11f3e2"}
Sep 15 20:36:30.231606 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:30.231508    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-725e67-667c64b48c-kz47q"
Sep 15 20:36:30.231606 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:30.231543    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-725e67-667c64b48c-kz47q" event={"ID":"fe36ea96-bfb6-4fd1-a260-a27872453328","Type":"ContainerDied","Data":"cc6f3b14fe78ae9c8511c25d07cba9b4033b26b8ee7a2bc9f4258c889e1a33f2"}
Sep 15 20:36:30.238959 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:30.238939    2576 scope.go:117] "RemoveContainer" containerID="694e6158c90dd5d733fdf973f79b35923c49673644ee32e88b5a78e2d18adeb8"
Sep 15 20:36:30.239220 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:36:30.239199    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"694e6158c90dd5d733fdf973f79b35923c49673644ee32e88b5a78e2d18adeb8\": container with ID starting with 694e6158c90dd5d733fdf973f79b35923c49673644ee32e88b5a78e2d18adeb8 not found: ID does not exist" containerID="694e6158c90dd5d733fdf973f79b35923c49673644ee32e88b5a78e2d18adeb8"
Sep 15 20:36:30.239272 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:30.239228    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"694e6158c90dd5d733fdf973f79b35923c49673644ee32e88b5a78e2d18adeb8"} err="failed to get container status \"694e6158c90dd5d733fdf973f79b35923c49673644ee32e88b5a78e2d18adeb8\": rpc error: code = NotFound desc = could not find container \"694e6158c90dd5d733fdf973f79b35923c49673644ee32e88b5a78e2d18adeb8\": container with ID starting with 694e6158c90dd5d733fdf973f79b35923c49673644ee32e88b5a78e2d18adeb8 not found: ID does not exist"
Sep 15 20:36:30.239272 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:30.239245    2576 scope.go:117] "RemoveContainer" containerID="b6ca48352debbd5b57ef5427bb047c0091fb29d9c150c4bd232265c1ae11f3e2"
Sep 15 20:36:30.247222 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:30.247202    2576 scope.go:117] "RemoveContainer" containerID="b6ca48352debbd5b57ef5427bb047c0091fb29d9c150c4bd232265c1ae11f3e2"
Sep 15 20:36:30.247502 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:36:30.247481    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6ca48352debbd5b57ef5427bb047c0091fb29d9c150c4bd232265c1ae11f3e2\": container with ID starting with b6ca48352debbd5b57ef5427bb047c0091fb29d9c150c4bd232265c1ae11f3e2 not found: ID does not exist" containerID="b6ca48352debbd5b57ef5427bb047c0091fb29d9c150c4bd232265c1ae11f3e2"
Sep 15 20:36:30.247568 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:30.247508    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6ca48352debbd5b57ef5427bb047c0091fb29d9c150c4bd232265c1ae11f3e2"} err="failed to get container status \"b6ca48352debbd5b57ef5427bb047c0091fb29d9c150c4bd232265c1ae11f3e2\": rpc error: code = NotFound desc = could not find container \"b6ca48352debbd5b57ef5427bb047c0091fb29d9c150c4bd232265c1ae11f3e2\": container with ID starting with b6ca48352debbd5b57ef5427bb047c0091fb29d9c150c4bd232265c1ae11f3e2 not found: ID does not exist"
Sep 15 20:36:30.249225 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:30.249208    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-725e67-667855bfcd-cgzbp"]
Sep 15 20:36:30.252978 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:30.252959    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-725e67-667855bfcd-cgzbp"]
Sep 15 20:36:30.266371 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:30.266351    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-725e67-667c64b48c-kz47q"]
Sep 15 20:36:30.270333 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:30.270314    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-725e67-667c64b48c-kz47q"]
Sep 15 20:36:30.691787 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:30.691711    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:36:31.474073 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:31.474044    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:36:32.144382 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:32.144349    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53826b73-d03f-4af1-a2aa-33825571d570" path="/var/lib/kubelet/pods/53826b73-d03f-4af1-a2aa-33825571d570/volumes"
Sep 15 20:36:32.144737 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:32.144728    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe36ea96-bfb6-4fd1-a260-a27872453328" path="/var/lib/kubelet/pods/fe36ea96-bfb6-4fd1-a260-a27872453328/volumes"
Sep 15 20:36:33.666968 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:33.666904    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr" podUID="2b77ce5f-d129-4eb5-966c-aae7ad0c1d04" containerName="main" containerID="cri-o://19a6ac4951b04e24475f0dcb0d4c2bf6b8d70819978193cfa22111f3fe98cd7e" gracePeriod=45
Sep 15 20:36:33.669727 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:33.669694    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf" podUID="b1af4267-cc2b-4a2a-8141-ef0a64aa29cf" containerName="main" containerID="cri-o://2121178f8b3af9ded180cd6c23ba21f74aea07af881727a55b1cd9d076efbb3a" gracePeriod=45
Sep 15 20:36:33.681127 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:33.681093    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8" podUID="e5731ba6-6c6a-4538-88b0-3c2e7533b337" containerName="main" containerID="cri-o://8907815a5b11ae7451f41fdaadfef82d7ec40992049d14aa59f03c8cf5786f36" gracePeriod=45
Sep 15 20:36:33.925471 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:33.925409    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf"
Sep 15 20:36:33.968170 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:33.968136    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr"
Sep 15 20:36:33.979742 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:33.979714    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8"
Sep 15 20:36:34.021961 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.021913    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vv69d\" (UniqueName: \"kubernetes.io/projected/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-kube-api-access-vv69d\") pod \"b1af4267-cc2b-4a2a-8141-ef0a64aa29cf\" (UID: \"b1af4267-cc2b-4a2a-8141-ef0a64aa29cf\") "
Sep 15 20:36:34.022190 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.022162    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5731ba6-6c6a-4538-88b0-3c2e7533b337-home\") pod \"e5731ba6-6c6a-4538-88b0-3c2e7533b337\" (UID: \"e5731ba6-6c6a-4538-88b0-3c2e7533b337\") "
Sep 15 20:36:34.022337 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.022209    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-dshm\") pod \"2b77ce5f-d129-4eb5-966c-aae7ad0c1d04\" (UID: \"2b77ce5f-d129-4eb5-966c-aae7ad0c1d04\") "
Sep 15 20:36:34.022337 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.022247    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-model-cache\") pod \"b1af4267-cc2b-4a2a-8141-ef0a64aa29cf\" (UID: \"b1af4267-cc2b-4a2a-8141-ef0a64aa29cf\") "
Sep 15 20:36:34.022337 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.022294    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-model-cache\") pod \"2b77ce5f-d129-4eb5-966c-aae7ad0c1d04\" (UID: \"2b77ce5f-d129-4eb5-966c-aae7ad0c1d04\") "
Sep 15 20:36:34.022337 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.022313    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5731ba6-6c6a-4538-88b0-3c2e7533b337-model-cache\") pod \"e5731ba6-6c6a-4538-88b0-3c2e7533b337\" (UID: \"e5731ba6-6c6a-4538-88b0-3c2e7533b337\") "
Sep 15 20:36:34.022553 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.022348    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-dshm\") pod \"b1af4267-cc2b-4a2a-8141-ef0a64aa29cf\" (UID: \"b1af4267-cc2b-4a2a-8141-ef0a64aa29cf\") "
Sep 15 20:36:34.022553 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.022395    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5731ba6-6c6a-4538-88b0-3c2e7533b337-dshm\") pod \"e5731ba6-6c6a-4538-88b0-3c2e7533b337\" (UID: \"e5731ba6-6c6a-4538-88b0-3c2e7533b337\") "
Sep 15 20:36:34.022553 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.022420    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5731ba6-6c6a-4538-88b0-3c2e7533b337-tmp-dir\") pod \"e5731ba6-6c6a-4538-88b0-3c2e7533b337\" (UID: \"e5731ba6-6c6a-4538-88b0-3c2e7533b337\") "
Sep 15 20:36:34.022553 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.022469    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5731ba6-6c6a-4538-88b0-3c2e7533b337-tls-certs\") pod \"e5731ba6-6c6a-4538-88b0-3c2e7533b337\" (UID: \"e5731ba6-6c6a-4538-88b0-3c2e7533b337\") "
Sep 15 20:36:34.022553 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.022494    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2xqzm\" (UniqueName: \"kubernetes.io/projected/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-kube-api-access-2xqzm\") pod \"2b77ce5f-d129-4eb5-966c-aae7ad0c1d04\" (UID: \"2b77ce5f-d129-4eb5-966c-aae7ad0c1d04\") "
Sep 15 20:36:34.022553 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.022548    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-tls-certs\") pod \"b1af4267-cc2b-4a2a-8141-ef0a64aa29cf\" (UID: \"b1af4267-cc2b-4a2a-8141-ef0a64aa29cf\") "
Sep 15 20:36:34.022886 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.022566    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-model-cache" (OuterVolumeSpecName: "model-cache") pod "2b77ce5f-d129-4eb5-966c-aae7ad0c1d04" (UID: "2b77ce5f-d129-4eb5-966c-aae7ad0c1d04"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:36:34.022886 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.022580    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-tmp-dir\") pod \"b1af4267-cc2b-4a2a-8141-ef0a64aa29cf\" (UID: \"b1af4267-cc2b-4a2a-8141-ef0a64aa29cf\") "
Sep 15 20:36:34.022886 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.022610    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8rln\" (UniqueName: \"kubernetes.io/projected/e5731ba6-6c6a-4538-88b0-3c2e7533b337-kube-api-access-g8rln\") pod \"e5731ba6-6c6a-4538-88b0-3c2e7533b337\" (UID: \"e5731ba6-6c6a-4538-88b0-3c2e7533b337\") "
Sep 15 20:36:34.022886 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.022642    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-home\") pod \"b1af4267-cc2b-4a2a-8141-ef0a64aa29cf\" (UID: \"b1af4267-cc2b-4a2a-8141-ef0a64aa29cf\") "
Sep 15 20:36:34.022886 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.022689    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-home\") pod \"2b77ce5f-d129-4eb5-966c-aae7ad0c1d04\" (UID: \"2b77ce5f-d129-4eb5-966c-aae7ad0c1d04\") "
Sep 15 20:36:34.022886 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.022722    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-tls-certs\") pod \"2b77ce5f-d129-4eb5-966c-aae7ad0c1d04\" (UID: \"2b77ce5f-d129-4eb5-966c-aae7ad0c1d04\") "
Sep 15 20:36:34.022886 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.022754    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-tmp-dir\") pod \"2b77ce5f-d129-4eb5-966c-aae7ad0c1d04\" (UID: \"2b77ce5f-d129-4eb5-966c-aae7ad0c1d04\") "
Sep 15 20:36:34.022886 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.022759    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5731ba6-6c6a-4538-88b0-3c2e7533b337-model-cache" (OuterVolumeSpecName: "model-cache") pod "e5731ba6-6c6a-4538-88b0-3c2e7533b337" (UID: "e5731ba6-6c6a-4538-88b0-3c2e7533b337"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:36:34.023342 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.022958    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-home" (OuterVolumeSpecName: "home") pod "b1af4267-cc2b-4a2a-8141-ef0a64aa29cf" (UID: "b1af4267-cc2b-4a2a-8141-ef0a64aa29cf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:36:34.023342 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.023104    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b1af4267-cc2b-4a2a-8141-ef0a64aa29cf" (UID: "b1af4267-cc2b-4a2a-8141-ef0a64aa29cf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:36:34.023342 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.023120    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-home\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:36:34.023342 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.023137    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-model-cache\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:36:34.023342 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.023151    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5731ba6-6c6a-4538-88b0-3c2e7533b337-model-cache\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:36:34.023342 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.023263    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5731ba6-6c6a-4538-88b0-3c2e7533b337-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e5731ba6-6c6a-4538-88b0-3c2e7533b337" (UID: "e5731ba6-6c6a-4538-88b0-3c2e7533b337"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:36:34.023342 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.023324    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2b77ce5f-d129-4eb5-966c-aae7ad0c1d04" (UID: "2b77ce5f-d129-4eb5-966c-aae7ad0c1d04"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:36:34.023712 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.023352    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5731ba6-6c6a-4538-88b0-3c2e7533b337-home" (OuterVolumeSpecName: "home") pod "e5731ba6-6c6a-4538-88b0-3c2e7533b337" (UID: "e5731ba6-6c6a-4538-88b0-3c2e7533b337"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:36:34.023712 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.023517    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-home" (OuterVolumeSpecName: "home") pod "2b77ce5f-d129-4eb5-966c-aae7ad0c1d04" (UID: "2b77ce5f-d129-4eb5-966c-aae7ad0c1d04"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:36:34.023712 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.023590    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-model-cache" (OuterVolumeSpecName: "model-cache") pod "b1af4267-cc2b-4a2a-8141-ef0a64aa29cf" (UID: "b1af4267-cc2b-4a2a-8141-ef0a64aa29cf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:36:34.026059 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.025821    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5731ba6-6c6a-4538-88b0-3c2e7533b337-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e5731ba6-6c6a-4538-88b0-3c2e7533b337" (UID: "e5731ba6-6c6a-4538-88b0-3c2e7533b337"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:36:34.026059 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.025838    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-dshm" (OuterVolumeSpecName: "dshm") pod "2b77ce5f-d129-4eb5-966c-aae7ad0c1d04" (UID: "2b77ce5f-d129-4eb5-966c-aae7ad0c1d04"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:36:34.026059 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.025877    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-kube-api-access-vv69d" (OuterVolumeSpecName: "kube-api-access-vv69d") pod "b1af4267-cc2b-4a2a-8141-ef0a64aa29cf" (UID: "b1af4267-cc2b-4a2a-8141-ef0a64aa29cf"). InnerVolumeSpecName "kube-api-access-vv69d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:36:34.026059 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.025999    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-dshm" (OuterVolumeSpecName: "dshm") pod "b1af4267-cc2b-4a2a-8141-ef0a64aa29cf" (UID: "b1af4267-cc2b-4a2a-8141-ef0a64aa29cf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:36:34.026346 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.026129    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5731ba6-6c6a-4538-88b0-3c2e7533b337-kube-api-access-g8rln" (OuterVolumeSpecName: "kube-api-access-g8rln") pod "e5731ba6-6c6a-4538-88b0-3c2e7533b337" (UID: "e5731ba6-6c6a-4538-88b0-3c2e7533b337"). InnerVolumeSpecName "kube-api-access-g8rln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:36:34.026410 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.026344    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2b77ce5f-d129-4eb5-966c-aae7ad0c1d04" (UID: "2b77ce5f-d129-4eb5-966c-aae7ad0c1d04"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:36:34.026777 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.026746    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b1af4267-cc2b-4a2a-8141-ef0a64aa29cf" (UID: "b1af4267-cc2b-4a2a-8141-ef0a64aa29cf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:36:34.027231 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.027205    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-kube-api-access-2xqzm" (OuterVolumeSpecName: "kube-api-access-2xqzm") pod "2b77ce5f-d129-4eb5-966c-aae7ad0c1d04" (UID: "2b77ce5f-d129-4eb5-966c-aae7ad0c1d04"). InnerVolumeSpecName "kube-api-access-2xqzm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:36:34.027320 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.027286    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5731ba6-6c6a-4538-88b0-3c2e7533b337-dshm" (OuterVolumeSpecName: "dshm") pod "e5731ba6-6c6a-4538-88b0-3c2e7533b337" (UID: "e5731ba6-6c6a-4538-88b0-3c2e7533b337"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:36:34.123783 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.123753    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vv69d\" (UniqueName: \"kubernetes.io/projected/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-kube-api-access-vv69d\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:36:34.123783 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.123779    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5731ba6-6c6a-4538-88b0-3c2e7533b337-home\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:36:34.123783 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.123789    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-dshm\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:36:34.124016 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.123797    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-model-cache\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:36:34.124016 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.123805    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-dshm\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:36:34.124016 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.123812    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5731ba6-6c6a-4538-88b0-3c2e7533b337-dshm\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:36:34.124016 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.123819    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5731ba6-6c6a-4538-88b0-3c2e7533b337-tmp-dir\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:36:34.124016 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.123827    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5731ba6-6c6a-4538-88b0-3c2e7533b337-tls-certs\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:36:34.124016 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.123834    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xqzm\" (UniqueName: \"kubernetes.io/projected/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-kube-api-access-2xqzm\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:36:34.124016 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.123843    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-tls-certs\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:36:34.124016 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.123850    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf-tmp-dir\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:36:34.124016 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.123857    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8rln\" (UniqueName: \"kubernetes.io/projected/e5731ba6-6c6a-4538-88b0-3c2e7533b337-kube-api-access-g8rln\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:36:34.124016 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.123864    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-home\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:36:34.124016 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.123871    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-tls-certs\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:36:34.124016 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.123878    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04-tmp-dir\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:36:34.259482 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.259442    2576 generic.go:356] "Generic (PLEG): container finished" podID="2b77ce5f-d129-4eb5-966c-aae7ad0c1d04" containerID="19a6ac4951b04e24475f0dcb0d4c2bf6b8d70819978193cfa22111f3fe98cd7e" exitCode=0
Sep 15 20:36:34.259646 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.259493    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr"
Sep 15 20:36:34.259646 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.259522    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr" event={"ID":"2b77ce5f-d129-4eb5-966c-aae7ad0c1d04","Type":"ContainerDied","Data":"19a6ac4951b04e24475f0dcb0d4c2bf6b8d70819978193cfa22111f3fe98cd7e"}
Sep 15 20:36:34.259646 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.259561    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr" event={"ID":"2b77ce5f-d129-4eb5-966c-aae7ad0c1d04","Type":"ContainerDied","Data":"a09fef78107215983939a82a37e87d93b6f2b7d3432254565c75c1ef73e54f90"}
Sep 15 20:36:34.259646 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.259583    2576 scope.go:117] "RemoveContainer" containerID="19a6ac4951b04e24475f0dcb0d4c2bf6b8d70819978193cfa22111f3fe98cd7e"
Sep 15 20:36:34.260934 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.260901    2576 generic.go:356] "Generic (PLEG): container finished" podID="e5731ba6-6c6a-4538-88b0-3c2e7533b337" containerID="8907815a5b11ae7451f41fdaadfef82d7ec40992049d14aa59f03c8cf5786f36" exitCode=0
Sep 15 20:36:34.261025 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.261002    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8"
Sep 15 20:36:34.261098 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.260996    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8" event={"ID":"e5731ba6-6c6a-4538-88b0-3c2e7533b337","Type":"ContainerDied","Data":"8907815a5b11ae7451f41fdaadfef82d7ec40992049d14aa59f03c8cf5786f36"}
Sep 15 20:36:34.261151 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.261125    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8" event={"ID":"e5731ba6-6c6a-4538-88b0-3c2e7533b337","Type":"ContainerDied","Data":"a175dd6606d425bf61849b55a71b694f52285f50bbd6055fed1d749335f6c0e3"}
Sep 15 20:36:34.262419 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.262398    2576 generic.go:356] "Generic (PLEG): container finished" podID="b1af4267-cc2b-4a2a-8141-ef0a64aa29cf" containerID="2121178f8b3af9ded180cd6c23ba21f74aea07af881727a55b1cd9d076efbb3a" exitCode=0
Sep 15 20:36:34.262513 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.262435    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf"
Sep 15 20:36:34.262513 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.262482    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf" event={"ID":"b1af4267-cc2b-4a2a-8141-ef0a64aa29cf","Type":"ContainerDied","Data":"2121178f8b3af9ded180cd6c23ba21f74aea07af881727a55b1cd9d076efbb3a"}
Sep 15 20:36:34.262601 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.262520    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf" event={"ID":"b1af4267-cc2b-4a2a-8141-ef0a64aa29cf","Type":"ContainerDied","Data":"62cf0b0a89d6d2ddeb753d036eb483c08deda861d9f286610db43349de47d348"}
Sep 15 20:36:34.269657 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.269642    2576 scope.go:117] "RemoveContainer" containerID="19a6ac4951b04e24475f0dcb0d4c2bf6b8d70819978193cfa22111f3fe98cd7e"
Sep 15 20:36:34.269931 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:36:34.269903    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19a6ac4951b04e24475f0dcb0d4c2bf6b8d70819978193cfa22111f3fe98cd7e\": container with ID starting with 19a6ac4951b04e24475f0dcb0d4c2bf6b8d70819978193cfa22111f3fe98cd7e not found: ID does not exist" containerID="19a6ac4951b04e24475f0dcb0d4c2bf6b8d70819978193cfa22111f3fe98cd7e"
Sep 15 20:36:34.269973 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.269950    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19a6ac4951b04e24475f0dcb0d4c2bf6b8d70819978193cfa22111f3fe98cd7e"} err="failed to get container status \"19a6ac4951b04e24475f0dcb0d4c2bf6b8d70819978193cfa22111f3fe98cd7e\": rpc error: code = NotFound desc = could not find container \"19a6ac4951b04e24475f0dcb0d4c2bf6b8d70819978193cfa22111f3fe98cd7e\": container with ID starting with 19a6ac4951b04e24475f0dcb0d4c2bf6b8d70819978193cfa22111f3fe98cd7e not found: ID does not exist"
Sep 15 20:36:34.269973 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.269967    2576 scope.go:117] "RemoveContainer" containerID="8907815a5b11ae7451f41fdaadfef82d7ec40992049d14aa59f03c8cf5786f36"
Sep 15 20:36:34.277694 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.277672    2576 scope.go:117] "RemoveContainer" containerID="8907815a5b11ae7451f41fdaadfef82d7ec40992049d14aa59f03c8cf5786f36"
Sep 15 20:36:34.278021 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:36:34.277995    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8907815a5b11ae7451f41fdaadfef82d7ec40992049d14aa59f03c8cf5786f36\": container with ID starting with 8907815a5b11ae7451f41fdaadfef82d7ec40992049d14aa59f03c8cf5786f36 not found: ID does not exist" containerID="8907815a5b11ae7451f41fdaadfef82d7ec40992049d14aa59f03c8cf5786f36"
Sep 15 20:36:34.278123 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.278023    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8907815a5b11ae7451f41fdaadfef82d7ec40992049d14aa59f03c8cf5786f36"} err="failed to get container status \"8907815a5b11ae7451f41fdaadfef82d7ec40992049d14aa59f03c8cf5786f36\": rpc error: code = NotFound desc = could not find container \"8907815a5b11ae7451f41fdaadfef82d7ec40992049d14aa59f03c8cf5786f36\": container with ID starting with 8907815a5b11ae7451f41fdaadfef82d7ec40992049d14aa59f03c8cf5786f36 not found: ID does not exist"
Sep 15 20:36:34.278123 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.278046    2576 scope.go:117] "RemoveContainer" containerID="2121178f8b3af9ded180cd6c23ba21f74aea07af881727a55b1cd9d076efbb3a"
Sep 15 20:36:34.280339 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.280322    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8"]
Sep 15 20:36:34.283616 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.283595    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-725e67/premium-w4-725e67-kserve-64fd8b959b-jkfl8"]
Sep 15 20:36:34.286226 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.286210    2576 scope.go:117] "RemoveContainer" containerID="2121178f8b3af9ded180cd6c23ba21f74aea07af881727a55b1cd9d076efbb3a"
Sep 15 20:36:34.286468 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:36:34.286449    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2121178f8b3af9ded180cd6c23ba21f74aea07af881727a55b1cd9d076efbb3a\": container with ID starting with 2121178f8b3af9ded180cd6c23ba21f74aea07af881727a55b1cd9d076efbb3a not found: ID does not exist" containerID="2121178f8b3af9ded180cd6c23ba21f74aea07af881727a55b1cd9d076efbb3a"
Sep 15 20:36:34.286527 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.286475    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2121178f8b3af9ded180cd6c23ba21f74aea07af881727a55b1cd9d076efbb3a"} err="failed to get container status \"2121178f8b3af9ded180cd6c23ba21f74aea07af881727a55b1cd9d076efbb3a\": rpc error: code = NotFound desc = could not find container \"2121178f8b3af9ded180cd6c23ba21f74aea07af881727a55b1cd9d076efbb3a\": container with ID starting with 2121178f8b3af9ded180cd6c23ba21f74aea07af881727a55b1cd9d076efbb3a not found: ID does not exist"
Sep 15 20:36:34.292868 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.292845    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr"]
Sep 15 20:36:34.296677 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.296657    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-725e67/distinct-2-w4-725e67-kserve-c988ccb4-6s6gr"]
Sep 15 20:36:34.305311 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.305289    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf"]
Sep 15 20:36:34.309666 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:34.309644    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-725e67/distinct-w4-725e67-kserve-84475bc795-hpscf"]
Sep 15 20:36:36.092466 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:36.092435    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:36:36.145331 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:36.145304    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b77ce5f-d129-4eb5-966c-aae7ad0c1d04" path="/var/lib/kubelet/pods/2b77ce5f-d129-4eb5-966c-aae7ad0c1d04/volumes"
Sep 15 20:36:36.145716 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:36.145698    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1af4267-cc2b-4a2a-8141-ef0a64aa29cf" path="/var/lib/kubelet/pods/b1af4267-cc2b-4a2a-8141-ef0a64aa29cf/volumes"
Sep 15 20:36:36.146075 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:36.146062    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5731ba6-6c6a-4538-88b0-3c2e7533b337" path="/var/lib/kubelet/pods/e5731ba6-6c6a-4538-88b0-3c2e7533b337/volumes"
Sep 15 20:36:36.163104 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:36.163083    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:36:36.367438 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:36.367359    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:36:46.267808 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:46.267773    2576 scope.go:117] "RemoveContainer" containerID="727daf89a2e99319d782e28b8d9344aef8d230c8b6639a983502ddb2ca8a2d6e"
Sep 15 20:36:46.276442 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:46.276424    2576 scope.go:117] "RemoveContainer" containerID="9c0b569a594486293beff734330a826039445450b866c1afb1d8d0d39d196ae0"
Sep 15 20:36:46.284132 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:46.284116    2576 scope.go:117] "RemoveContainer" containerID="e16ee77b04c81cd67a4150e432da2ffb3559c8c701035eb0c7b70c1320f51951"
Sep 15 20:36:46.291224 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:46.291210    2576 scope.go:117] "RemoveContainer" containerID="fa9d1479d0b11c1c2a80a2faa476f6966ad4b0a3d8398be61a045e5c2ef79c53"
Sep 15 20:36:46.298269 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:46.298252    2576 scope.go:117] "RemoveContainer" containerID="cca46fb54b7264b8c7f8a65b2a0d26a061f4009bb97d8291ff5cf48491ee0787"
Sep 15 20:36:54.940629 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.940597    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns"]
Sep 15 20:36:54.941043 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941026    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76689ad7-ea6b-44cf-bb1b-2190a608e81e" containerName="payload-pre-processing"
Sep 15 20:36:54.941107 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941045    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="76689ad7-ea6b-44cf-bb1b-2190a608e81e" containerName="payload-pre-processing"
Sep 15 20:36:54.941107 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941057    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cd2c1ab-b253-40fd-b902-5de5943ab450" containerName="main"
Sep 15 20:36:54.941107 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941062    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cd2c1ab-b253-40fd-b902-5de5943ab450" containerName="main"
Sep 15 20:36:54.941107 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941078    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68aa3f0c-93b1-44e7-b21e-3e5353baea2a" containerName="main"
Sep 15 20:36:54.941107 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941084    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="68aa3f0c-93b1-44e7-b21e-3e5353baea2a" containerName="main"
Sep 15 20:36:54.941107 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941093    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53826b73-d03f-4af1-a2aa-33825571d570" containerName="payload-pre-processing"
Sep 15 20:36:54.941107 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941098    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="53826b73-d03f-4af1-a2aa-33825571d570" containerName="payload-pre-processing"
Sep 15 20:36:54.941107 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941108    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8aa6c347-9591-490f-bc74-4ee372e2db2f" containerName="main"
Sep 15 20:36:54.941361 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941113    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8aa6c347-9591-490f-bc74-4ee372e2db2f" containerName="main"
Sep 15 20:36:54.941361 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941119    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5731ba6-6c6a-4538-88b0-3c2e7533b337" containerName="main"
Sep 15 20:36:54.941361 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941123    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5731ba6-6c6a-4538-88b0-3c2e7533b337" containerName="main"
Sep 15 20:36:54.941361 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941131    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95418b12-f771-4e7c-836e-bb5e04eac166" containerName="payload-pre-processing"
Sep 15 20:36:54.941361 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941136    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="95418b12-f771-4e7c-836e-bb5e04eac166" containerName="payload-pre-processing"
Sep 15 20:36:54.941361 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941141    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7912229-70e1-4ec7-8c05-7f756a8a9411" containerName="payload-processing"
Sep 15 20:36:54.941361 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941146    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7912229-70e1-4ec7-8c05-7f756a8a9411" containerName="payload-processing"
Sep 15 20:36:54.941361 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941151    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b77ce5f-d129-4eb5-966c-aae7ad0c1d04" containerName="main"
Sep 15 20:36:54.941361 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941156    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b77ce5f-d129-4eb5-966c-aae7ad0c1d04" containerName="main"
Sep 15 20:36:54.941361 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941166    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1af4267-cc2b-4a2a-8141-ef0a64aa29cf" containerName="main"
Sep 15 20:36:54.941361 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941173    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1af4267-cc2b-4a2a-8141-ef0a64aa29cf" containerName="main"
Sep 15 20:36:54.941361 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941183    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe36ea96-bfb6-4fd1-a260-a27872453328" containerName="payload-processing"
Sep 15 20:36:54.941361 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941190    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe36ea96-bfb6-4fd1-a260-a27872453328" containerName="payload-processing"
Sep 15 20:36:54.941361 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941198    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efca3da3-ac68-4bfb-aa1c-4672b38c5894" containerName="revoke-keys"
Sep 15 20:36:54.941361 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941206    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="efca3da3-ac68-4bfb-aa1c-4672b38c5894" containerName="revoke-keys"
Sep 15 20:36:54.941361 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941266    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe36ea96-bfb6-4fd1-a260-a27872453328" containerName="payload-processing"
Sep 15 20:36:54.941361 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941274    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="76689ad7-ea6b-44cf-bb1b-2190a608e81e" containerName="payload-pre-processing"
Sep 15 20:36:54.941361 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941282    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1af4267-cc2b-4a2a-8141-ef0a64aa29cf" containerName="main"
Sep 15 20:36:54.941361 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941290    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="68aa3f0c-93b1-44e7-b21e-3e5353baea2a" containerName="main"
Sep 15 20:36:54.941361 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941296    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cd2c1ab-b253-40fd-b902-5de5943ab450" containerName="main"
Sep 15 20:36:54.941361 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941302    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b77ce5f-d129-4eb5-966c-aae7ad0c1d04" containerName="main"
Sep 15 20:36:54.941361 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941307    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="efca3da3-ac68-4bfb-aa1c-4672b38c5894" containerName="revoke-keys"
Sep 15 20:36:54.941361 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941314    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="95418b12-f771-4e7c-836e-bb5e04eac166" containerName="payload-pre-processing"
Sep 15 20:36:54.941361 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941319    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7912229-70e1-4ec7-8c05-7f756a8a9411" containerName="payload-processing"
Sep 15 20:36:54.941361 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941323    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5731ba6-6c6a-4538-88b0-3c2e7533b337" containerName="main"
Sep 15 20:36:54.941361 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941329    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8aa6c347-9591-490f-bc74-4ee372e2db2f" containerName="main"
Sep 15 20:36:54.941361 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.941334    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="53826b73-d03f-4af1-a2aa-33825571d570" containerName="payload-pre-processing"
Sep 15 20:36:54.944264 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.944248    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns"
Sep 15 20:36:54.947160 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.947135    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-e80b38\"/\"kube-root-ca.crt\""
Sep 15 20:36:54.947160 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.947147    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-e80b38\"/\"default-dockercfg-lgr7w\""
Sep 15 20:36:54.947341 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.947180    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-e80b38\"/\"sub-test-model-e80b38-kserve-self-signed-certs\""
Sep 15 20:36:54.947341 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.947147    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-e80b38\"/\"openshift-service-ca.crt\""
Sep 15 20:36:54.950579 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.950559    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns"]
Sep 15 20:36:54.961214 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:54.961187    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:36:55.010552 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:55.010523    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77f98017-9373-4eae-8f03-22d290026f31-model-cache\") pod \"sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns\" (UID: \"77f98017-9373-4eae-8f03-22d290026f31\") " pod="ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns"
Sep 15 20:36:55.010675 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:55.010571    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77f98017-9373-4eae-8f03-22d290026f31-dshm\") pod \"sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns\" (UID: \"77f98017-9373-4eae-8f03-22d290026f31\") " pod="ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns"
Sep 15 20:36:55.010675 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:55.010606    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77f98017-9373-4eae-8f03-22d290026f31-tmp-dir\") pod \"sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns\" (UID: \"77f98017-9373-4eae-8f03-22d290026f31\") " pod="ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns"
Sep 15 20:36:55.010675 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:55.010633    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxvzn\" (UniqueName: \"kubernetes.io/projected/77f98017-9373-4eae-8f03-22d290026f31-kube-api-access-sxvzn\") pod \"sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns\" (UID: \"77f98017-9373-4eae-8f03-22d290026f31\") " pod="ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns"
Sep 15 20:36:55.010675 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:55.010658    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77f98017-9373-4eae-8f03-22d290026f31-tls-certs\") pod \"sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns\" (UID: \"77f98017-9373-4eae-8f03-22d290026f31\") " pod="ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns"
Sep 15 20:36:55.010808 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:55.010731    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77f98017-9373-4eae-8f03-22d290026f31-home\") pod \"sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns\" (UID: \"77f98017-9373-4eae-8f03-22d290026f31\") " pod="ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns"
Sep 15 20:36:55.111585 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:55.111547    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77f98017-9373-4eae-8f03-22d290026f31-dshm\") pod \"sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns\" (UID: \"77f98017-9373-4eae-8f03-22d290026f31\") " pod="ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns"
Sep 15 20:36:55.111720 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:55.111591    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77f98017-9373-4eae-8f03-22d290026f31-tmp-dir\") pod \"sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns\" (UID: \"77f98017-9373-4eae-8f03-22d290026f31\") " pod="ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns"
Sep 15 20:36:55.111720 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:55.111626    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxvzn\" (UniqueName: \"kubernetes.io/projected/77f98017-9373-4eae-8f03-22d290026f31-kube-api-access-sxvzn\") pod \"sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns\" (UID: \"77f98017-9373-4eae-8f03-22d290026f31\") " pod="ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns"
Sep 15 20:36:55.111720 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:55.111663    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77f98017-9373-4eae-8f03-22d290026f31-tls-certs\") pod \"sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns\" (UID: \"77f98017-9373-4eae-8f03-22d290026f31\") " pod="ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns"
Sep 15 20:36:55.111720 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:55.111715    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77f98017-9373-4eae-8f03-22d290026f31-home\") pod \"sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns\" (UID: \"77f98017-9373-4eae-8f03-22d290026f31\") " pod="ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns"
Sep 15 20:36:55.111996 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:55.111765    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77f98017-9373-4eae-8f03-22d290026f31-model-cache\") pod \"sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns\" (UID: \"77f98017-9373-4eae-8f03-22d290026f31\") " pod="ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns"
Sep 15 20:36:55.112113 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:55.112088    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77f98017-9373-4eae-8f03-22d290026f31-tmp-dir\") pod \"sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns\" (UID: \"77f98017-9373-4eae-8f03-22d290026f31\") " pod="ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns"
Sep 15 20:36:55.112161 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:55.112097    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77f98017-9373-4eae-8f03-22d290026f31-model-cache\") pod \"sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns\" (UID: \"77f98017-9373-4eae-8f03-22d290026f31\") " pod="ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns"
Sep 15 20:36:55.112161 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:55.112131    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77f98017-9373-4eae-8f03-22d290026f31-home\") pod \"sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns\" (UID: \"77f98017-9373-4eae-8f03-22d290026f31\") " pod="ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns"
Sep 15 20:36:55.113870 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:55.113850    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77f98017-9373-4eae-8f03-22d290026f31-dshm\") pod \"sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns\" (UID: \"77f98017-9373-4eae-8f03-22d290026f31\") " pod="ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns"
Sep 15 20:36:55.114169 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:55.114151    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77f98017-9373-4eae-8f03-22d290026f31-tls-certs\") pod \"sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns\" (UID: \"77f98017-9373-4eae-8f03-22d290026f31\") " pod="ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns"
Sep 15 20:36:55.119500 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:55.119480    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxvzn\" (UniqueName: \"kubernetes.io/projected/77f98017-9373-4eae-8f03-22d290026f31-kube-api-access-sxvzn\") pod \"sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns\" (UID: \"77f98017-9373-4eae-8f03-22d290026f31\") " pod="ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns"
Sep 15 20:36:55.253767 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:55.253719    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns"
Sep 15 20:36:55.522264 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:55.522239    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns"]
Sep 15 20:36:55.524130 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:36:55.524101    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77f98017_9373_4eae_8f03_22d290026f31.slice/crio-1c0062b801d166f1028ed72bb4b85c9436c847bc0f6905b11a747d4aed9147fc WatchSource:0}: Error finding container 1c0062b801d166f1028ed72bb4b85c9436c847bc0f6905b11a747d4aed9147fc: Status 404 returned error can't find the container with id 1c0062b801d166f1028ed72bb4b85c9436c847bc0f6905b11a747d4aed9147fc
Sep 15 20:36:56.206051 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:56.206006    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:36:56.307120 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:56.307056    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:36:56.379429 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:56.379394    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns" event={"ID":"77f98017-9373-4eae-8f03-22d290026f31","Type":"ContainerStarted","Data":"00e456af710df5c40b9200017d78c84e9154b9e008e14701bf134d417fafaa40"}
Sep 15 20:36:56.379599 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:56.379441    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns" event={"ID":"77f98017-9373-4eae-8f03-22d290026f31","Type":"ContainerStarted","Data":"1c0062b801d166f1028ed72bb4b85c9436c847bc0f6905b11a747d4aed9147fc"}
Sep 15 20:36:56.397170 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:56.397112    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns" podStartSLOduration=2.39709774 podStartE2EDuration="2.39709774s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:36:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:36:56.395624675 +0000 UTC m=+1930.884089150" watchObservedRunningTime="2026-09-15 20:36:56.39709774 +0000 UTC m=+1930.885562265"
Sep 15 20:36:56.400585 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:36:56.400564    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:37:02.000824 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:02.000785    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:37:05.253878 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:05.253845    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns"
Sep 15 20:37:05.253878 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:05.253889    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns"
Sep 15 20:37:05.266356 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:05.266330    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns"
Sep 15 20:37:05.435183 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:05.435154    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns"
Sep 15 20:37:34.634939 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:34.634889    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:37:35.732165 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:35.732132    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns"]
Sep 15 20:37:35.752285 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:35.752245    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:37:50.751814 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:50.751700    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns" podUID="77f98017-9373-4eae-8f03-22d290026f31" containerName="main" containerID="cri-o://00e456af710df5c40b9200017d78c84e9154b9e008e14701bf134d417fafaa40" gracePeriod=45
Sep 15 20:37:51.005274 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:51.005220    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns"
Sep 15 20:37:51.097193 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:51.097165    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77f98017-9373-4eae-8f03-22d290026f31-tmp-dir\") pod \"77f98017-9373-4eae-8f03-22d290026f31\" (UID: \"77f98017-9373-4eae-8f03-22d290026f31\") "
Sep 15 20:37:51.097340 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:51.097218    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxvzn\" (UniqueName: \"kubernetes.io/projected/77f98017-9373-4eae-8f03-22d290026f31-kube-api-access-sxvzn\") pod \"77f98017-9373-4eae-8f03-22d290026f31\" (UID: \"77f98017-9373-4eae-8f03-22d290026f31\") "
Sep 15 20:37:51.097340 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:51.097255    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77f98017-9373-4eae-8f03-22d290026f31-tls-certs\") pod \"77f98017-9373-4eae-8f03-22d290026f31\" (UID: \"77f98017-9373-4eae-8f03-22d290026f31\") "
Sep 15 20:37:51.097340 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:51.097276    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77f98017-9373-4eae-8f03-22d290026f31-dshm\") pod \"77f98017-9373-4eae-8f03-22d290026f31\" (UID: \"77f98017-9373-4eae-8f03-22d290026f31\") "
Sep 15 20:37:51.097340 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:51.097310    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77f98017-9373-4eae-8f03-22d290026f31-home\") pod \"77f98017-9373-4eae-8f03-22d290026f31\" (UID: \"77f98017-9373-4eae-8f03-22d290026f31\") "
Sep 15 20:37:51.097507 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:51.097366    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77f98017-9373-4eae-8f03-22d290026f31-model-cache\") pod \"77f98017-9373-4eae-8f03-22d290026f31\" (UID: \"77f98017-9373-4eae-8f03-22d290026f31\") "
Sep 15 20:37:51.097507 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:51.097477    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77f98017-9373-4eae-8f03-22d290026f31-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "77f98017-9373-4eae-8f03-22d290026f31" (UID: "77f98017-9373-4eae-8f03-22d290026f31"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:37:51.097638 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:51.097615    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77f98017-9373-4eae-8f03-22d290026f31-tmp-dir\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:37:51.097745 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:51.097650    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77f98017-9373-4eae-8f03-22d290026f31-home" (OuterVolumeSpecName: "home") pod "77f98017-9373-4eae-8f03-22d290026f31" (UID: "77f98017-9373-4eae-8f03-22d290026f31"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:37:51.097745 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:51.097668    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77f98017-9373-4eae-8f03-22d290026f31-model-cache" (OuterVolumeSpecName: "model-cache") pod "77f98017-9373-4eae-8f03-22d290026f31" (UID: "77f98017-9373-4eae-8f03-22d290026f31"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:37:51.099560 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:51.099538    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77f98017-9373-4eae-8f03-22d290026f31-kube-api-access-sxvzn" (OuterVolumeSpecName: "kube-api-access-sxvzn") pod "77f98017-9373-4eae-8f03-22d290026f31" (UID: "77f98017-9373-4eae-8f03-22d290026f31"). InnerVolumeSpecName "kube-api-access-sxvzn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:37:51.099560 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:51.099541    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77f98017-9373-4eae-8f03-22d290026f31-dshm" (OuterVolumeSpecName: "dshm") pod "77f98017-9373-4eae-8f03-22d290026f31" (UID: "77f98017-9373-4eae-8f03-22d290026f31"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:37:51.099687 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:51.099567    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77f98017-9373-4eae-8f03-22d290026f31-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "77f98017-9373-4eae-8f03-22d290026f31" (UID: "77f98017-9373-4eae-8f03-22d290026f31"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:37:51.198334 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:51.198309    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxvzn\" (UniqueName: \"kubernetes.io/projected/77f98017-9373-4eae-8f03-22d290026f31-kube-api-access-sxvzn\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:37:51.198441 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:51.198337    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77f98017-9373-4eae-8f03-22d290026f31-tls-certs\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:37:51.198441 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:51.198350    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77f98017-9373-4eae-8f03-22d290026f31-dshm\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:37:51.198441 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:51.198361    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77f98017-9373-4eae-8f03-22d290026f31-home\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:37:51.198441 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:51.198372    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77f98017-9373-4eae-8f03-22d290026f31-model-cache\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:37:51.653352 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:51.653319    2576 generic.go:356] "Generic (PLEG): container finished" podID="77f98017-9373-4eae-8f03-22d290026f31" containerID="00e456af710df5c40b9200017d78c84e9154b9e008e14701bf134d417fafaa40" exitCode=0
Sep 15 20:37:51.653352 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:51.653354    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns"
Sep 15 20:37:51.653598 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:51.653373    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns" event={"ID":"77f98017-9373-4eae-8f03-22d290026f31","Type":"ContainerDied","Data":"00e456af710df5c40b9200017d78c84e9154b9e008e14701bf134d417fafaa40"}
Sep 15 20:37:51.653598 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:51.653409    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns" event={"ID":"77f98017-9373-4eae-8f03-22d290026f31","Type":"ContainerDied","Data":"1c0062b801d166f1028ed72bb4b85c9436c847bc0f6905b11a747d4aed9147fc"}
Sep 15 20:37:51.653598 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:51.653432    2576 scope.go:117] "RemoveContainer" containerID="00e456af710df5c40b9200017d78c84e9154b9e008e14701bf134d417fafaa40"
Sep 15 20:37:51.662627 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:51.662612    2576 scope.go:117] "RemoveContainer" containerID="00e456af710df5c40b9200017d78c84e9154b9e008e14701bf134d417fafaa40"
Sep 15 20:37:51.662860 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:37:51.662841    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00e456af710df5c40b9200017d78c84e9154b9e008e14701bf134d417fafaa40\": container with ID starting with 00e456af710df5c40b9200017d78c84e9154b9e008e14701bf134d417fafaa40 not found: ID does not exist" containerID="00e456af710df5c40b9200017d78c84e9154b9e008e14701bf134d417fafaa40"
Sep 15 20:37:51.662913 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:51.662869    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00e456af710df5c40b9200017d78c84e9154b9e008e14701bf134d417fafaa40"} err="failed to get container status \"00e456af710df5c40b9200017d78c84e9154b9e008e14701bf134d417fafaa40\": rpc error: code = NotFound desc = could not find container \"00e456af710df5c40b9200017d78c84e9154b9e008e14701bf134d417fafaa40\": container with ID starting with 00e456af710df5c40b9200017d78c84e9154b9e008e14701bf134d417fafaa40 not found: ID does not exist"
Sep 15 20:37:51.672799 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:51.672775    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns"]
Sep 15 20:37:51.676309 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:51.676289    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-e80b38/sub-test-model-e80b38-kserve-5cf9df4b6c-wfqns"]
Sep 15 20:37:52.143966 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:52.143910    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77f98017-9373-4eae-8f03-22d290026f31" path="/var/lib/kubelet/pods/77f98017-9373-4eae-8f03-22d290026f31/volumes"
Sep 15 20:37:55.836642 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:55.836610    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp"]
Sep 15 20:37:55.837209 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:55.837189    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77f98017-9373-4eae-8f03-22d290026f31" containerName="main"
Sep 15 20:37:55.837326 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:55.837212    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="77f98017-9373-4eae-8f03-22d290026f31" containerName="main"
Sep 15 20:37:55.837326 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:55.837291    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="77f98017-9373-4eae-8f03-22d290026f31" containerName="main"
Sep 15 20:37:55.841517 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:55.841496    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp"
Sep 15 20:37:55.843895 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:55.843872    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-92702a\"/\"openshift-service-ca.crt\""
Sep 15 20:37:55.844799 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:55.844777    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-92702a\"/\"kube-root-ca.crt\""
Sep 15 20:37:55.844799 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:55.844794    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-92702a\"/\"rate-test-model-92702a-kserve-self-signed-certs\""
Sep 15 20:37:55.844957 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:55.844803    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-92702a\"/\"default-dockercfg-fcw5j\""
Sep 15 20:37:55.848989 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:55.848966    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp"]
Sep 15 20:37:55.863337 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:55.863315    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:37:55.937031 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:55.937012    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/163d0c15-7517-47a5-8264-1bbb17347e94-dshm\") pod \"rate-test-model-92702a-kserve-685576c-pg8dp\" (UID: \"163d0c15-7517-47a5-8264-1bbb17347e94\") " pod="ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp"
Sep 15 20:37:55.937131 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:55.937043    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/163d0c15-7517-47a5-8264-1bbb17347e94-tls-certs\") pod \"rate-test-model-92702a-kserve-685576c-pg8dp\" (UID: \"163d0c15-7517-47a5-8264-1bbb17347e94\") " pod="ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp"
Sep 15 20:37:55.937131 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:55.937090    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bg79m\" (UniqueName: \"kubernetes.io/projected/163d0c15-7517-47a5-8264-1bbb17347e94-kube-api-access-bg79m\") pod \"rate-test-model-92702a-kserve-685576c-pg8dp\" (UID: \"163d0c15-7517-47a5-8264-1bbb17347e94\") " pod="ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp"
Sep 15 20:37:55.937131 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:55.937108    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/163d0c15-7517-47a5-8264-1bbb17347e94-home\") pod \"rate-test-model-92702a-kserve-685576c-pg8dp\" (UID: \"163d0c15-7517-47a5-8264-1bbb17347e94\") " pod="ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp"
Sep 15 20:37:55.937238 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:55.937135    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/163d0c15-7517-47a5-8264-1bbb17347e94-tmp-dir\") pod \"rate-test-model-92702a-kserve-685576c-pg8dp\" (UID: \"163d0c15-7517-47a5-8264-1bbb17347e94\") " pod="ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp"
Sep 15 20:37:55.937238 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:55.937191    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/163d0c15-7517-47a5-8264-1bbb17347e94-model-cache\") pod \"rate-test-model-92702a-kserve-685576c-pg8dp\" (UID: \"163d0c15-7517-47a5-8264-1bbb17347e94\") " pod="ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp"
Sep 15 20:37:56.038524 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:56.038499    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/163d0c15-7517-47a5-8264-1bbb17347e94-tmp-dir\") pod \"rate-test-model-92702a-kserve-685576c-pg8dp\" (UID: \"163d0c15-7517-47a5-8264-1bbb17347e94\") " pod="ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp"
Sep 15 20:37:56.038636 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:56.038529    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/163d0c15-7517-47a5-8264-1bbb17347e94-model-cache\") pod \"rate-test-model-92702a-kserve-685576c-pg8dp\" (UID: \"163d0c15-7517-47a5-8264-1bbb17347e94\") " pod="ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp"
Sep 15 20:37:56.038636 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:56.038571    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/163d0c15-7517-47a5-8264-1bbb17347e94-dshm\") pod \"rate-test-model-92702a-kserve-685576c-pg8dp\" (UID: \"163d0c15-7517-47a5-8264-1bbb17347e94\") " pod="ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp"
Sep 15 20:37:56.038636 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:56.038597    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/163d0c15-7517-47a5-8264-1bbb17347e94-tls-certs\") pod \"rate-test-model-92702a-kserve-685576c-pg8dp\" (UID: \"163d0c15-7517-47a5-8264-1bbb17347e94\") " pod="ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp"
Sep 15 20:37:56.038771 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:56.038637    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bg79m\" (UniqueName: \"kubernetes.io/projected/163d0c15-7517-47a5-8264-1bbb17347e94-kube-api-access-bg79m\") pod \"rate-test-model-92702a-kserve-685576c-pg8dp\" (UID: \"163d0c15-7517-47a5-8264-1bbb17347e94\") " pod="ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp"
Sep 15 20:37:56.038771 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:56.038661    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/163d0c15-7517-47a5-8264-1bbb17347e94-home\") pod \"rate-test-model-92702a-kserve-685576c-pg8dp\" (UID: \"163d0c15-7517-47a5-8264-1bbb17347e94\") " pod="ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp"
Sep 15 20:37:56.038989 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:56.038967    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/163d0c15-7517-47a5-8264-1bbb17347e94-model-cache\") pod \"rate-test-model-92702a-kserve-685576c-pg8dp\" (UID: \"163d0c15-7517-47a5-8264-1bbb17347e94\") " pod="ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp"
Sep 15 20:37:56.039040 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:56.039000    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/163d0c15-7517-47a5-8264-1bbb17347e94-tmp-dir\") pod \"rate-test-model-92702a-kserve-685576c-pg8dp\" (UID: \"163d0c15-7517-47a5-8264-1bbb17347e94\") " pod="ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp"
Sep 15 20:37:56.039081 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:56.039039    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/163d0c15-7517-47a5-8264-1bbb17347e94-home\") pod \"rate-test-model-92702a-kserve-685576c-pg8dp\" (UID: \"163d0c15-7517-47a5-8264-1bbb17347e94\") " pod="ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp"
Sep 15 20:37:56.040850 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:56.040832    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/163d0c15-7517-47a5-8264-1bbb17347e94-dshm\") pod \"rate-test-model-92702a-kserve-685576c-pg8dp\" (UID: \"163d0c15-7517-47a5-8264-1bbb17347e94\") " pod="ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp"
Sep 15 20:37:56.041270 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:56.041244    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/163d0c15-7517-47a5-8264-1bbb17347e94-tls-certs\") pod \"rate-test-model-92702a-kserve-685576c-pg8dp\" (UID: \"163d0c15-7517-47a5-8264-1bbb17347e94\") " pod="ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp"
Sep 15 20:37:56.045726 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:56.045700    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bg79m\" (UniqueName: \"kubernetes.io/projected/163d0c15-7517-47a5-8264-1bbb17347e94-kube-api-access-bg79m\") pod \"rate-test-model-92702a-kserve-685576c-pg8dp\" (UID: \"163d0c15-7517-47a5-8264-1bbb17347e94\") " pod="ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp"
Sep 15 20:37:56.152602 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:56.152534    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp"
Sep 15 20:37:56.424085 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:56.424060    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp"]
Sep 15 20:37:56.425975 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:37:56.425943    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod163d0c15_7517_47a5_8264_1bbb17347e94.slice/crio-fe512c2d435c5c5cc4ab16601045680d264290e851d9e01d15553f79b28c3d88 WatchSource:0}: Error finding container fe512c2d435c5c5cc4ab16601045680d264290e851d9e01d15553f79b28c3d88: Status 404 returned error can't find the container with id fe512c2d435c5c5cc4ab16601045680d264290e851d9e01d15553f79b28c3d88
Sep 15 20:37:56.679644 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:56.679559    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp" event={"ID":"163d0c15-7517-47a5-8264-1bbb17347e94","Type":"ContainerStarted","Data":"8b042dbd9af86d35c0b41a6a9765627a15120125cd98037da23f362ea2e92abc"}
Sep 15 20:37:56.679644 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:56.679600    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp" event={"ID":"163d0c15-7517-47a5-8264-1bbb17347e94","Type":"ContainerStarted","Data":"fe512c2d435c5c5cc4ab16601045680d264290e851d9e01d15553f79b28c3d88"}
Sep 15 20:37:56.694247 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:37:56.694190    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp" podStartSLOduration=1.694177452 podStartE2EDuration="1.694177452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:37:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:37:56.694147809 +0000 UTC m=+1991.182612282" watchObservedRunningTime="2026-09-15 20:37:56.694177452 +0000 UTC m=+1991.182641927"
Sep 15 20:38:06.152970 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:38:06.152937    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp"
Sep 15 20:38:06.152970 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:38:06.152977    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp"
Sep 15 20:38:06.165282 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:38:06.165256    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp"
Sep 15 20:38:06.739330 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:38:06.739295    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp"
Sep 15 20:38:30.351421 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:38:30.351383    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:38:39.175417 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:38:39.175382    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:38:39.275228 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:38:39.275194    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:39:10.422639 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:10.422562    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp"]
Sep 15 20:39:10.460284 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:10.460253    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:39:11.576818 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:11.576785    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:39:12.000507 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:12.000479    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:39:12.307219 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:12.307144    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:39:12.875643 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:12.875609    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:39:15.555299 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:15.555258    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:39:15.810699 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:15.810611    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:39:15.909395 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:15.909354    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:39:16.359067 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:16.359036    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:39:17.021884 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:17.021853    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:39:17.089942 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:17.089895    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:39:17.189431 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:17.189399    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:39:17.480211 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:17.480180    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:39:17.576988 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:17.576947    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:39:17.581299 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:17.581264    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c005f8-6ff49f8558-sst7q"]
Sep 15 20:39:17.585115 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:17.585094    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c005f8-6ff49f8558-sst7q"
Sep 15 20:39:17.587375 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:17.587355    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-c005f8\""
Sep 15 20:39:17.587490 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:17.587356    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-c005f8-dockercfg-nnsj9\""
Sep 15 20:39:17.592535 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:17.592503    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c005f8-6ff49f8558-sst7q"]
Sep 15 20:39:17.654428 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:17.654400    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbc7e8b8-7c60-4918-95b2-18acc06e2edb-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-c005f8-6ff49f8558-sst7q\" (UID: \"fbc7e8b8-7c60-4918-95b2-18acc06e2edb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c005f8-6ff49f8558-sst7q"
Sep 15 20:39:17.654577 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:17.654505    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldvl6\" (UniqueName: \"kubernetes.io/projected/fbc7e8b8-7c60-4918-95b2-18acc06e2edb-kube-api-access-ldvl6\") pod \"payload-pre-processing-e2e-ipp-a-c005f8-6ff49f8558-sst7q\" (UID: \"fbc7e8b8-7c60-4918-95b2-18acc06e2edb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c005f8-6ff49f8558-sst7q"
Sep 15 20:39:17.755885 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:17.755804    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbc7e8b8-7c60-4918-95b2-18acc06e2edb-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-c005f8-6ff49f8558-sst7q\" (UID: \"fbc7e8b8-7c60-4918-95b2-18acc06e2edb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c005f8-6ff49f8558-sst7q"
Sep 15 20:39:17.756061 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:17.755951    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldvl6\" (UniqueName: \"kubernetes.io/projected/fbc7e8b8-7c60-4918-95b2-18acc06e2edb-kube-api-access-ldvl6\") pod \"payload-pre-processing-e2e-ipp-a-c005f8-6ff49f8558-sst7q\" (UID: \"fbc7e8b8-7c60-4918-95b2-18acc06e2edb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c005f8-6ff49f8558-sst7q"
Sep 15 20:39:17.756515 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:17.756496    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbc7e8b8-7c60-4918-95b2-18acc06e2edb-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-c005f8-6ff49f8558-sst7q\" (UID: \"fbc7e8b8-7c60-4918-95b2-18acc06e2edb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c005f8-6ff49f8558-sst7q"
Sep 15 20:39:17.771557 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:17.771526    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldvl6\" (UniqueName: \"kubernetes.io/projected/fbc7e8b8-7c60-4918-95b2-18acc06e2edb-kube-api-access-ldvl6\") pod \"payload-pre-processing-e2e-ipp-a-c005f8-6ff49f8558-sst7q\" (UID: \"fbc7e8b8-7c60-4918-95b2-18acc06e2edb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c005f8-6ff49f8558-sst7q"
Sep 15 20:39:17.897234 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:17.897201    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c005f8-6ff49f8558-sst7q"
Sep 15 20:39:18.167486 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:18.167453    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c005f8-6ff49f8558-sst7q"]
Sep 15 20:39:18.170521 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:39:18.170490    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbc7e8b8_7c60_4918_95b2_18acc06e2edb.slice/crio-7236618cfc4b602f5d5a92756dcea28deef87773d8c85692ce580cdbdae2ef80 WatchSource:0}: Error finding container 7236618cfc4b602f5d5a92756dcea28deef87773d8c85692ce580cdbdae2ef80: Status 404 returned error can't find the container with id 7236618cfc4b602f5d5a92756dcea28deef87773d8c85692ce580cdbdae2ef80
Sep 15 20:39:19.077864 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:19.077828    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c005f8-6ff49f8558-sst7q" event={"ID":"fbc7e8b8-7c60-4918-95b2-18acc06e2edb","Type":"ContainerStarted","Data":"bc68d9d5f984f8b070d69de5ce56b809348a3185dfc729b06e01ec8f91ee88b3"}
Sep 15 20:39:19.078067 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:19.077908    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c005f8-6ff49f8558-sst7q" event={"ID":"fbc7e8b8-7c60-4918-95b2-18acc06e2edb","Type":"ContainerStarted","Data":"7236618cfc4b602f5d5a92756dcea28deef87773d8c85692ce580cdbdae2ef80"}
Sep 15 20:39:19.078067 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:19.077968    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c005f8-6ff49f8558-sst7q"
Sep 15 20:39:19.094330 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:19.094281    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c005f8-6ff49f8558-sst7q" podStartSLOduration=2.094267454 podStartE2EDuration="2.094267454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:39:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:39:19.092035949 +0000 UTC m=+2073.580500425" watchObservedRunningTime="2026-09-15 20:39:19.094267454 +0000 UTC m=+2073.582731929"
Sep 15 20:39:25.440948 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:25.440863    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp" podUID="163d0c15-7517-47a5-8264-1bbb17347e94" containerName="main" containerID="cri-o://8b042dbd9af86d35c0b41a6a9765627a15120125cd98037da23f362ea2e92abc" gracePeriod=45
Sep 15 20:39:25.681825 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:25.681800    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp"
Sep 15 20:39:25.726651 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:25.726628    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/163d0c15-7517-47a5-8264-1bbb17347e94-model-cache\") pod \"163d0c15-7517-47a5-8264-1bbb17347e94\" (UID: \"163d0c15-7517-47a5-8264-1bbb17347e94\") "
Sep 15 20:39:25.726818 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:25.726661    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bg79m\" (UniqueName: \"kubernetes.io/projected/163d0c15-7517-47a5-8264-1bbb17347e94-kube-api-access-bg79m\") pod \"163d0c15-7517-47a5-8264-1bbb17347e94\" (UID: \"163d0c15-7517-47a5-8264-1bbb17347e94\") "
Sep 15 20:39:25.726818 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:25.726692    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/163d0c15-7517-47a5-8264-1bbb17347e94-home\") pod \"163d0c15-7517-47a5-8264-1bbb17347e94\" (UID: \"163d0c15-7517-47a5-8264-1bbb17347e94\") "
Sep 15 20:39:25.726818 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:25.726794    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/163d0c15-7517-47a5-8264-1bbb17347e94-dshm\") pod \"163d0c15-7517-47a5-8264-1bbb17347e94\" (UID: \"163d0c15-7517-47a5-8264-1bbb17347e94\") "
Sep 15 20:39:25.727041 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:25.726840    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/163d0c15-7517-47a5-8264-1bbb17347e94-tls-certs\") pod \"163d0c15-7517-47a5-8264-1bbb17347e94\" (UID: \"163d0c15-7517-47a5-8264-1bbb17347e94\") "
Sep 15 20:39:25.727041 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:25.726904    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/163d0c15-7517-47a5-8264-1bbb17347e94-tmp-dir\") pod \"163d0c15-7517-47a5-8264-1bbb17347e94\" (UID: \"163d0c15-7517-47a5-8264-1bbb17347e94\") "
Sep 15 20:39:25.727041 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:25.726956    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/163d0c15-7517-47a5-8264-1bbb17347e94-model-cache" (OuterVolumeSpecName: "model-cache") pod "163d0c15-7517-47a5-8264-1bbb17347e94" (UID: "163d0c15-7517-47a5-8264-1bbb17347e94"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:39:25.727041 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:25.726967    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/163d0c15-7517-47a5-8264-1bbb17347e94-home" (OuterVolumeSpecName: "home") pod "163d0c15-7517-47a5-8264-1bbb17347e94" (UID: "163d0c15-7517-47a5-8264-1bbb17347e94"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:39:25.727260 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:25.727236    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/163d0c15-7517-47a5-8264-1bbb17347e94-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "163d0c15-7517-47a5-8264-1bbb17347e94" (UID: "163d0c15-7517-47a5-8264-1bbb17347e94"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:39:25.727324 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:25.727254    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/163d0c15-7517-47a5-8264-1bbb17347e94-model-cache\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:39:25.727324 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:25.727276    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/163d0c15-7517-47a5-8264-1bbb17347e94-home\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:39:25.728877 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:25.728851    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/163d0c15-7517-47a5-8264-1bbb17347e94-dshm" (OuterVolumeSpecName: "dshm") pod "163d0c15-7517-47a5-8264-1bbb17347e94" (UID: "163d0c15-7517-47a5-8264-1bbb17347e94"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:39:25.728877 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:25.728850    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/163d0c15-7517-47a5-8264-1bbb17347e94-kube-api-access-bg79m" (OuterVolumeSpecName: "kube-api-access-bg79m") pod "163d0c15-7517-47a5-8264-1bbb17347e94" (UID: "163d0c15-7517-47a5-8264-1bbb17347e94"). InnerVolumeSpecName "kube-api-access-bg79m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:39:25.729062 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:25.728884    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/163d0c15-7517-47a5-8264-1bbb17347e94-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "163d0c15-7517-47a5-8264-1bbb17347e94" (UID: "163d0c15-7517-47a5-8264-1bbb17347e94"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:39:25.828427 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:25.828401    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bg79m\" (UniqueName: \"kubernetes.io/projected/163d0c15-7517-47a5-8264-1bbb17347e94-kube-api-access-bg79m\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:39:25.828427 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:25.828424    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/163d0c15-7517-47a5-8264-1bbb17347e94-dshm\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:39:25.828585 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:25.828434    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/163d0c15-7517-47a5-8264-1bbb17347e94-tls-certs\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:39:25.828585 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:25.828443    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/163d0c15-7517-47a5-8264-1bbb17347e94-tmp-dir\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:39:26.114334 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:26.114247    2576 generic.go:356] "Generic (PLEG): container finished" podID="163d0c15-7517-47a5-8264-1bbb17347e94" containerID="8b042dbd9af86d35c0b41a6a9765627a15120125cd98037da23f362ea2e92abc" exitCode=0
Sep 15 20:39:26.114334 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:26.114298    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp"
Sep 15 20:39:26.114563 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:26.114338    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp" event={"ID":"163d0c15-7517-47a5-8264-1bbb17347e94","Type":"ContainerDied","Data":"8b042dbd9af86d35c0b41a6a9765627a15120125cd98037da23f362ea2e92abc"}
Sep 15 20:39:26.114563 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:26.114385    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp" event={"ID":"163d0c15-7517-47a5-8264-1bbb17347e94","Type":"ContainerDied","Data":"fe512c2d435c5c5cc4ab16601045680d264290e851d9e01d15553f79b28c3d88"}
Sep 15 20:39:26.114563 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:26.114408    2576 scope.go:117] "RemoveContainer" containerID="8b042dbd9af86d35c0b41a6a9765627a15120125cd98037da23f362ea2e92abc"
Sep 15 20:39:26.123955 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:26.123934    2576 scope.go:117] "RemoveContainer" containerID="8b042dbd9af86d35c0b41a6a9765627a15120125cd98037da23f362ea2e92abc"
Sep 15 20:39:26.124219 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:39:26.124201    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b042dbd9af86d35c0b41a6a9765627a15120125cd98037da23f362ea2e92abc\": container with ID starting with 8b042dbd9af86d35c0b41a6a9765627a15120125cd98037da23f362ea2e92abc not found: ID does not exist" containerID="8b042dbd9af86d35c0b41a6a9765627a15120125cd98037da23f362ea2e92abc"
Sep 15 20:39:26.124270 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:26.124230    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b042dbd9af86d35c0b41a6a9765627a15120125cd98037da23f362ea2e92abc"} err="failed to get container status \"8b042dbd9af86d35c0b41a6a9765627a15120125cd98037da23f362ea2e92abc\": rpc error: code = NotFound desc = could not find container \"8b042dbd9af86d35c0b41a6a9765627a15120125cd98037da23f362ea2e92abc\": container with ID starting with 8b042dbd9af86d35c0b41a6a9765627a15120125cd98037da23f362ea2e92abc not found: ID does not exist"
Sep 15 20:39:26.134530 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:26.134512    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp"]
Sep 15 20:39:26.144026 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:26.144007    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-92702a/rate-test-model-92702a-kserve-685576c-pg8dp"]
Sep 15 20:39:28.142644 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:28.142614    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="163d0c15-7517-47a5-8264-1bbb17347e94" path="/var/lib/kubelet/pods/163d0c15-7517-47a5-8264-1bbb17347e94/volumes"
Sep 15 20:39:30.086100 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:30.086069    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c005f8-6ff49f8558-sst7q"
Sep 15 20:39:39.782542 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:39.782502    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:39:40.709045 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:40.709010    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:39:40.913565 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:40.913533    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:39:41.229120 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:41.229086    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:39:41.467572 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:41.467540    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:39:41.784888 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:41.784854    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:39:41.986895 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:41.986838    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:39:42.092693 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:42.092613    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:39:44.265821 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:44.265783    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:39:44.948112 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:44.948074    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:39:45.051582 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:45.051554    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:39:45.148964 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:45.148911    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:39:45.931249 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:45.931214    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:39:46.753514 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:46.753462    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:39:47.634347 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:47.634316    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:39:47.735078 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:47.735031    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:39:47.785980 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:47.785952    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:39:48.070423 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:48.070393    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:39:48.135414 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:48.135377    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:39:48.166564 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:48.166533    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-8dc819-5fd6986f8b-fsn6l"]
Sep 15 20:39:48.166905 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:48.166893    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="163d0c15-7517-47a5-8264-1bbb17347e94" containerName="main"
Sep 15 20:39:48.166969 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:48.166907    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="163d0c15-7517-47a5-8264-1bbb17347e94" containerName="main"
Sep 15 20:39:48.167010 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:48.166989    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="163d0c15-7517-47a5-8264-1bbb17347e94" containerName="main"
Sep 15 20:39:48.171412 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:48.171396    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8dc819-5fd6986f8b-fsn6l"
Sep 15 20:39:48.173740 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:48.173718    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-8dc819\""
Sep 15 20:39:48.173857 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:48.173755    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-8dc819-dockercfg-p7lrc\""
Sep 15 20:39:48.180853 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:48.180825    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-8dc819-5fd6986f8b-fsn6l"]
Sep 15 20:39:48.204754 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:48.204725    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-8dc819-7477658649-kdccd"]
Sep 15 20:39:48.208378 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:48.208358    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-8dc819-7477658649-kdccd"
Sep 15 20:39:48.218885 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:48.218864    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-8dc819-7477658649-kdccd"]
Sep 15 20:39:48.327595 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:48.327514    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bg5p\" (UniqueName: \"kubernetes.io/projected/d996108e-e0b4-4379-959c-f816a6d82211-kube-api-access-2bg5p\") pod \"payload-processing-e2e-ipp-b-8dc819-7477658649-kdccd\" (UID: \"d996108e-e0b4-4379-959c-f816a6d82211\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-8dc819-7477658649-kdccd"
Sep 15 20:39:48.327746 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:48.327617    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d996108e-e0b4-4379-959c-f816a6d82211-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-8dc819-7477658649-kdccd\" (UID: \"d996108e-e0b4-4379-959c-f816a6d82211\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-8dc819-7477658649-kdccd"
Sep 15 20:39:48.327746 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:48.327691    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd141af0-4e29-40a2-8856-ea19c689ea5e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-8dc819-5fd6986f8b-fsn6l\" (UID: \"fd141af0-4e29-40a2-8856-ea19c689ea5e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8dc819-5fd6986f8b-fsn6l"
Sep 15 20:39:48.327746 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:48.327709    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpj6k\" (UniqueName: \"kubernetes.io/projected/fd141af0-4e29-40a2-8856-ea19c689ea5e-kube-api-access-fpj6k\") pod \"payload-pre-processing-e2e-ipp-b-8dc819-5fd6986f8b-fsn6l\" (UID: \"fd141af0-4e29-40a2-8856-ea19c689ea5e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8dc819-5fd6986f8b-fsn6l"
Sep 15 20:39:48.428812 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:48.428767    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d996108e-e0b4-4379-959c-f816a6d82211-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-8dc819-7477658649-kdccd\" (UID: \"d996108e-e0b4-4379-959c-f816a6d82211\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-8dc819-7477658649-kdccd"
Sep 15 20:39:48.429005 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:48.428848    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd141af0-4e29-40a2-8856-ea19c689ea5e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-8dc819-5fd6986f8b-fsn6l\" (UID: \"fd141af0-4e29-40a2-8856-ea19c689ea5e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8dc819-5fd6986f8b-fsn6l"
Sep 15 20:39:48.429005 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:48.428869    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpj6k\" (UniqueName: \"kubernetes.io/projected/fd141af0-4e29-40a2-8856-ea19c689ea5e-kube-api-access-fpj6k\") pod \"payload-pre-processing-e2e-ipp-b-8dc819-5fd6986f8b-fsn6l\" (UID: \"fd141af0-4e29-40a2-8856-ea19c689ea5e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8dc819-5fd6986f8b-fsn6l"
Sep 15 20:39:48.429005 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:48.428898    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bg5p\" (UniqueName: \"kubernetes.io/projected/d996108e-e0b4-4379-959c-f816a6d82211-kube-api-access-2bg5p\") pod \"payload-processing-e2e-ipp-b-8dc819-7477658649-kdccd\" (UID: \"d996108e-e0b4-4379-959c-f816a6d82211\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-8dc819-7477658649-kdccd"
Sep 15 20:39:48.429630 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:48.429601    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d996108e-e0b4-4379-959c-f816a6d82211-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-8dc819-7477658649-kdccd\" (UID: \"d996108e-e0b4-4379-959c-f816a6d82211\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-8dc819-7477658649-kdccd"
Sep 15 20:39:48.429761 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:48.429618    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd141af0-4e29-40a2-8856-ea19c689ea5e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-8dc819-5fd6986f8b-fsn6l\" (UID: \"fd141af0-4e29-40a2-8856-ea19c689ea5e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8dc819-5fd6986f8b-fsn6l"
Sep 15 20:39:48.436488 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:48.436467    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpj6k\" (UniqueName: \"kubernetes.io/projected/fd141af0-4e29-40a2-8856-ea19c689ea5e-kube-api-access-fpj6k\") pod \"payload-pre-processing-e2e-ipp-b-8dc819-5fd6986f8b-fsn6l\" (UID: \"fd141af0-4e29-40a2-8856-ea19c689ea5e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8dc819-5fd6986f8b-fsn6l"
Sep 15 20:39:48.436587 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:48.436513    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bg5p\" (UniqueName: \"kubernetes.io/projected/d996108e-e0b4-4379-959c-f816a6d82211-kube-api-access-2bg5p\") pod \"payload-processing-e2e-ipp-b-8dc819-7477658649-kdccd\" (UID: \"d996108e-e0b4-4379-959c-f816a6d82211\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-8dc819-7477658649-kdccd"
Sep 15 20:39:48.481583 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:48.481561    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8dc819-5fd6986f8b-fsn6l"
Sep 15 20:39:48.519653 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:48.519622    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-8dc819-7477658649-kdccd"
Sep 15 20:39:48.749442 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:48.749411    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-8dc819-5fd6986f8b-fsn6l"]
Sep 15 20:39:48.753339 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:39:48.753313    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd141af0_4e29_40a2_8856_ea19c689ea5e.slice/crio-3297992ce892fac7ee2cff3a396468b3c5e6a95ea3336e9372beeba5230d4e08 WatchSource:0}: Error finding container 3297992ce892fac7ee2cff3a396468b3c5e6a95ea3336e9372beeba5230d4e08: Status 404 returned error can't find the container with id 3297992ce892fac7ee2cff3a396468b3c5e6a95ea3336e9372beeba5230d4e08
Sep 15 20:39:48.805075 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:48.805047    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-8dc819-7477658649-kdccd"]
Sep 15 20:39:48.807971 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:39:48.807946    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd996108e_e0b4_4379_959c_f816a6d82211.slice/crio-fe2e65c94b99531e70cb26c07cae98f48a3dc5573b333b18ec618a2827880666 WatchSource:0}: Error finding container fe2e65c94b99531e70cb26c07cae98f48a3dc5573b333b18ec618a2827880666: Status 404 returned error can't find the container with id fe2e65c94b99531e70cb26c07cae98f48a3dc5573b333b18ec618a2827880666
Sep 15 20:39:49.234207 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:49.234159    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-8dc819-7477658649-kdccd" event={"ID":"d996108e-e0b4-4379-959c-f816a6d82211","Type":"ContainerStarted","Data":"d2b5e3b774a4be8f534ee43091f8900e8abdcb557fc4699bd58180761cbdc7c5"}
Sep 15 20:39:49.234414 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:49.234227    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-8dc819-7477658649-kdccd" event={"ID":"d996108e-e0b4-4379-959c-f816a6d82211","Type":"ContainerStarted","Data":"fe2e65c94b99531e70cb26c07cae98f48a3dc5573b333b18ec618a2827880666"}
Sep 15 20:39:49.234414 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:49.234352    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-8dc819-7477658649-kdccd"
Sep 15 20:39:49.235960 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:49.235913    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8dc819-5fd6986f8b-fsn6l" event={"ID":"fd141af0-4e29-40a2-8856-ea19c689ea5e","Type":"ContainerStarted","Data":"0bec450d3e0bf08d900b303295680adcc513cce69a81d1a390c035a8e8d2392e"}
Sep 15 20:39:49.236076 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:49.235980    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8dc819-5fd6986f8b-fsn6l" event={"ID":"fd141af0-4e29-40a2-8856-ea19c689ea5e","Type":"ContainerStarted","Data":"3297992ce892fac7ee2cff3a396468b3c5e6a95ea3336e9372beeba5230d4e08"}
Sep 15 20:39:49.236076 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:49.236025    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8dc819-5fd6986f8b-fsn6l"
Sep 15 20:39:49.251818 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:49.251774    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-8dc819-7477658649-kdccd" podStartSLOduration=1.251760006 podStartE2EDuration="1.251760006s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:39:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:39:49.250227468 +0000 UTC m=+2103.738691944" watchObservedRunningTime="2026-09-15 20:39:49.251760006 +0000 UTC m=+2103.740224482"
Sep 15 20:39:49.265621 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:39:49.265570    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8dc819-5fd6986f8b-fsn6l" podStartSLOduration=1.2655503860000001 podStartE2EDuration="1.265550386s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:39:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:39:49.263783942 +0000 UTC m=+2103.752248420" watchObservedRunningTime="2026-09-15 20:39:49.265550386 +0000 UTC m=+2103.754014863"
Sep 15 20:40:00.243766 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:00.243731    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8dc819-5fd6986f8b-fsn6l"
Sep 15 20:40:00.244160 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:00.243801    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-8dc819-7477658649-kdccd"
Sep 15 20:40:10.367204 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:10.367161    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:40:11.662644 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:11.662609    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:40:11.763347 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:11.763313    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:40:12.354305 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:12.354269    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:40:12.966342 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:12.966311    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:40:15.062257 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:15.062215    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:40:20.147338 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:20.147310    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-8dc819-7477658649-kdccd_d996108e-e0b4-4379-959c-f816a6d82211/payload-processing/0.log"
Sep 15 20:40:21.071255 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:21.071222    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:40:24.330976 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:24.330912    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v"]
Sep 15 20:40:24.334719 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:24.334698    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v"
Sep 15 20:40:24.337705 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:24.337683    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-c005f8\"/\"openshift-service-ca.crt\""
Sep 15 20:40:24.337864 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:24.337748    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-c005f8\"/\"default-dockercfg-krkqz\""
Sep 15 20:40:24.337864 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:24.337767    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-c005f8\"/\"ipp-route-c005f8-kserve-self-signed-certs\""
Sep 15 20:40:24.337864 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:24.337755    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-c005f8\"/\"kube-root-ca.crt\""
Sep 15 20:40:24.341868 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:24.341840    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v"]
Sep 15 20:40:24.356515 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:24.356486    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:40:24.456363 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:24.456330    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvfvc\" (UniqueName: \"kubernetes.io/projected/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-kube-api-access-bvfvc\") pod \"ipp-route-c005f8-kserve-549d77d459-tjk8v\" (UID: \"a1a79b2d-fbf2-46f5-97a5-3a7689069f89\") " pod="ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v"
Sep 15 20:40:24.456540 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:24.456374    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-dshm\") pod \"ipp-route-c005f8-kserve-549d77d459-tjk8v\" (UID: \"a1a79b2d-fbf2-46f5-97a5-3a7689069f89\") " pod="ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v"
Sep 15 20:40:24.456540 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:24.456428    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-model-cache\") pod \"ipp-route-c005f8-kserve-549d77d459-tjk8v\" (UID: \"a1a79b2d-fbf2-46f5-97a5-3a7689069f89\") " pod="ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v"
Sep 15 20:40:24.456540 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:24.456501    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-tls-certs\") pod \"ipp-route-c005f8-kserve-549d77d459-tjk8v\" (UID: \"a1a79b2d-fbf2-46f5-97a5-3a7689069f89\") " pod="ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v"
Sep 15 20:40:24.456540 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:24.456516    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-home\") pod \"ipp-route-c005f8-kserve-549d77d459-tjk8v\" (UID: \"a1a79b2d-fbf2-46f5-97a5-3a7689069f89\") " pod="ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v"
Sep 15 20:40:24.456540 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:24.456533    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-tmp-dir\") pod \"ipp-route-c005f8-kserve-549d77d459-tjk8v\" (UID: \"a1a79b2d-fbf2-46f5-97a5-3a7689069f89\") " pod="ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v"
Sep 15 20:40:24.557587 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:24.557477    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-tls-certs\") pod \"ipp-route-c005f8-kserve-549d77d459-tjk8v\" (UID: \"a1a79b2d-fbf2-46f5-97a5-3a7689069f89\") " pod="ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v"
Sep 15 20:40:24.557587 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:24.557511    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-home\") pod \"ipp-route-c005f8-kserve-549d77d459-tjk8v\" (UID: \"a1a79b2d-fbf2-46f5-97a5-3a7689069f89\") " pod="ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v"
Sep 15 20:40:24.557587 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:24.557538    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-tmp-dir\") pod \"ipp-route-c005f8-kserve-549d77d459-tjk8v\" (UID: \"a1a79b2d-fbf2-46f5-97a5-3a7689069f89\") " pod="ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v"
Sep 15 20:40:24.563035 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:24.557663    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvfvc\" (UniqueName: \"kubernetes.io/projected/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-kube-api-access-bvfvc\") pod \"ipp-route-c005f8-kserve-549d77d459-tjk8v\" (UID: \"a1a79b2d-fbf2-46f5-97a5-3a7689069f89\") " pod="ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v"
Sep 15 20:40:24.563035 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:24.557718    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-dshm\") pod \"ipp-route-c005f8-kserve-549d77d459-tjk8v\" (UID: \"a1a79b2d-fbf2-46f5-97a5-3a7689069f89\") " pod="ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v"
Sep 15 20:40:24.563035 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:24.557772    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-model-cache\") pod \"ipp-route-c005f8-kserve-549d77d459-tjk8v\" (UID: \"a1a79b2d-fbf2-46f5-97a5-3a7689069f89\") " pod="ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v"
Sep 15 20:40:24.563035 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:24.557888    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-home\") pod \"ipp-route-c005f8-kserve-549d77d459-tjk8v\" (UID: \"a1a79b2d-fbf2-46f5-97a5-3a7689069f89\") " pod="ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v"
Sep 15 20:40:24.563035 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:24.557950    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-tmp-dir\") pod \"ipp-route-c005f8-kserve-549d77d459-tjk8v\" (UID: \"a1a79b2d-fbf2-46f5-97a5-3a7689069f89\") " pod="ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v"
Sep 15 20:40:24.563035 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:24.558114    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-model-cache\") pod \"ipp-route-c005f8-kserve-549d77d459-tjk8v\" (UID: \"a1a79b2d-fbf2-46f5-97a5-3a7689069f89\") " pod="ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v"
Sep 15 20:40:24.563035 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:24.559781    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-dshm\") pod \"ipp-route-c005f8-kserve-549d77d459-tjk8v\" (UID: \"a1a79b2d-fbf2-46f5-97a5-3a7689069f89\") " pod="ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v"
Sep 15 20:40:24.563035 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:24.560147    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-tls-certs\") pod \"ipp-route-c005f8-kserve-549d77d459-tjk8v\" (UID: \"a1a79b2d-fbf2-46f5-97a5-3a7689069f89\") " pod="ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v"
Sep 15 20:40:24.565358 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:24.565335    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvfvc\" (UniqueName: \"kubernetes.io/projected/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-kube-api-access-bvfvc\") pod \"ipp-route-c005f8-kserve-549d77d459-tjk8v\" (UID: \"a1a79b2d-fbf2-46f5-97a5-3a7689069f89\") " pod="ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v"
Sep 15 20:40:24.647007 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:24.646951    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v"
Sep 15 20:40:24.916671 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:24.916602    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v"]
Sep 15 20:40:24.919860 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:40:24.919829    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda1a79b2d_fbf2_46f5_97a5_3a7689069f89.slice/crio-c03d57ef192d6a61065c5491c6f0760620188e4b64142111c5d7a4d794966064 WatchSource:0}: Error finding container c03d57ef192d6a61065c5491c6f0760620188e4b64142111c5d7a4d794966064: Status 404 returned error can't find the container with id c03d57ef192d6a61065c5491c6f0760620188e4b64142111c5d7a4d794966064
Sep 15 20:40:25.417084 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:25.417044    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v" event={"ID":"a1a79b2d-fbf2-46f5-97a5-3a7689069f89","Type":"ContainerStarted","Data":"155effbbaa9d0df80dda47bc88fede8422edd99de732a5cf9594b43c66f9833a"}
Sep 15 20:40:25.417084 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:25.417090    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v" event={"ID":"a1a79b2d-fbf2-46f5-97a5-3a7689069f89","Type":"ContainerStarted","Data":"c03d57ef192d6a61065c5491c6f0760620188e4b64142111c5d7a4d794966064"}
Sep 15 20:40:25.435439 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:25.435375    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v" podStartSLOduration=1.435358769 podStartE2EDuration="1.435358769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:40:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:40:25.435317015 +0000 UTC m=+2139.923781488" watchObservedRunningTime="2026-09-15 20:40:25.435358769 +0000 UTC m=+2139.923823246"
Sep 15 20:40:34.647330 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:34.647292    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v"
Sep 15 20:40:34.647731 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:34.647344    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v"
Sep 15 20:40:34.659984 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:34.659954    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v"
Sep 15 20:40:35.479367 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:35.479335    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v"
Sep 15 20:40:42.256448 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:42.256372    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:40:46.384908 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:46.384875    2576 scope.go:117] "RemoveContainer" containerID="efe19cdedc01f5bb9955b7c665a49dc3f53537f9622c634484d7309ba6c673b5"
Sep 15 20:40:46.393304 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:46.393284    2576 scope.go:117] "RemoveContainer" containerID="f90bfe49bb3acb6acbfcdb60cab9375fb1290d2df7a9c291cd841655e3c31dd1"
Sep 15 20:40:51.099796 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:51.099765    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-8dc819-7477658649-kdccd_d996108e-e0b4-4379-959c-f816a6d82211/payload-processing/0.log"
Sep 15 20:40:51.364178 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:51.364087    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:40:51.659265 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:51.659173    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:40:52.400653 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:52.400616    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:40:53.653020 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:53.652987    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:40:54.287418 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:54.287388    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:40:54.378328 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:54.378287    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:40:55.180854 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:55.180817    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:40:55.893357 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:55.893319    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:40:56.755964 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:56.755913    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:40:57.392752 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:57.392720    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:40:57.494443 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:57.494410    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:40:57.825180 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:57.825147    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:40:57.925484 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:57.925449    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:40:57.933357 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:57.933332    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a6b11e-5487b545f8-xfd64"]
Sep 15 20:40:57.936639 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:57.936623    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a6b11e-5487b545f8-xfd64"
Sep 15 20:40:57.938870 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:57.938849    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-a6b11e\""
Sep 15 20:40:57.939176 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:57.939157    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-a6b11e-dockercfg-4jrr2\""
Sep 15 20:40:57.954369 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:57.954348    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a6b11e-5487b545f8-xfd64"]
Sep 15 20:40:58.057168 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:58.057133    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e7e1ea1-9075-48bf-b45e-1d8904b40785-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-a6b11e-5487b545f8-xfd64\" (UID: \"3e7e1ea1-9075-48bf-b45e-1d8904b40785\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a6b11e-5487b545f8-xfd64"
Sep 15 20:40:58.057324 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:58.057194    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkpnm\" (UniqueName: \"kubernetes.io/projected/3e7e1ea1-9075-48bf-b45e-1d8904b40785-kube-api-access-hkpnm\") pod \"payload-pre-processing-e2e-ipp-cleanup-a6b11e-5487b545f8-xfd64\" (UID: \"3e7e1ea1-9075-48bf-b45e-1d8904b40785\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a6b11e-5487b545f8-xfd64"
Sep 15 20:40:58.158616 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:58.158542    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkpnm\" (UniqueName: \"kubernetes.io/projected/3e7e1ea1-9075-48bf-b45e-1d8904b40785-kube-api-access-hkpnm\") pod \"payload-pre-processing-e2e-ipp-cleanup-a6b11e-5487b545f8-xfd64\" (UID: \"3e7e1ea1-9075-48bf-b45e-1d8904b40785\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a6b11e-5487b545f8-xfd64"
Sep 15 20:40:58.158762 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:58.158645    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e7e1ea1-9075-48bf-b45e-1d8904b40785-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-a6b11e-5487b545f8-xfd64\" (UID: \"3e7e1ea1-9075-48bf-b45e-1d8904b40785\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a6b11e-5487b545f8-xfd64"
Sep 15 20:40:58.159368 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:58.159349    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e7e1ea1-9075-48bf-b45e-1d8904b40785-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-a6b11e-5487b545f8-xfd64\" (UID: \"3e7e1ea1-9075-48bf-b45e-1d8904b40785\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a6b11e-5487b545f8-xfd64"
Sep 15 20:40:58.166933 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:58.166895    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkpnm\" (UniqueName: \"kubernetes.io/projected/3e7e1ea1-9075-48bf-b45e-1d8904b40785-kube-api-access-hkpnm\") pod \"payload-pre-processing-e2e-ipp-cleanup-a6b11e-5487b545f8-xfd64\" (UID: \"3e7e1ea1-9075-48bf-b45e-1d8904b40785\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a6b11e-5487b545f8-xfd64"
Sep 15 20:40:58.246285 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:58.246262    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a6b11e-5487b545f8-xfd64"
Sep 15 20:40:58.515442 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:58.515416    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a6b11e-5487b545f8-xfd64"]
Sep 15 20:40:58.518074 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:40:58.518043    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e7e1ea1_9075_48bf_b45e_1d8904b40785.slice/crio-c3dead851ab00eff2053843183b0c8c527fcbab585d68e7c590a10f9d9df93b0 WatchSource:0}: Error finding container c3dead851ab00eff2053843183b0c8c527fcbab585d68e7c590a10f9d9df93b0: Status 404 returned error can't find the container with id c3dead851ab00eff2053843183b0c8c527fcbab585d68e7c590a10f9d9df93b0
Sep 15 20:40:58.587474 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:58.587449    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a6b11e-5487b545f8-xfd64" event={"ID":"3e7e1ea1-9075-48bf-b45e-1d8904b40785","Type":"ContainerStarted","Data":"64e660f78aca71449be602dd92dfe4485b896daa1111c250d24e4045b00c271c"}
Sep 15 20:40:58.587598 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:58.587502    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a6b11e-5487b545f8-xfd64"
Sep 15 20:40:58.587598 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:58.587520    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a6b11e-5487b545f8-xfd64" event={"ID":"3e7e1ea1-9075-48bf-b45e-1d8904b40785","Type":"ContainerStarted","Data":"c3dead851ab00eff2053843183b0c8c527fcbab585d68e7c590a10f9d9df93b0"}
Sep 15 20:40:58.602341 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:40:58.602300    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a6b11e-5487b545f8-xfd64" podStartSLOduration=1.602284907 podStartE2EDuration="1.602284907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:40:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:40:58.601269701 +0000 UTC m=+2173.089734175" watchObservedRunningTime="2026-09-15 20:40:58.602284907 +0000 UTC m=+2173.090749394"
Sep 15 20:41:01.475434 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:01.475404    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:01.527129 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:01.527101    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:09.594319 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:09.594282    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a6b11e-5487b545f8-xfd64"
Sep 15 20:41:19.307201 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:19.307161    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:19.410396 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:19.410363    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:19.978017 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:19.977978    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:19.999004 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:19.998972    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a6b11e-vck2d"]
Sep 15 20:41:20.002735 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:20.002718    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a6b11e-vck2d"
Sep 15 20:41:20.004979 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:20.004959    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r5lpt\""
Sep 15 20:41:20.005095 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:20.004975    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 20:41:20.009195 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:20.009168    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a6b11e-vck2d"]
Sep 15 20:41:20.157869 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:20.157837    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58a7c880-431e-4674-94db-821a970ffd52-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-a6b11e-vck2d\" (UID: \"58a7c880-431e-4674-94db-821a970ffd52\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a6b11e-vck2d"
Sep 15 20:41:20.258809 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:20.258717    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58a7c880-431e-4674-94db-821a970ffd52-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-a6b11e-vck2d\" (UID: \"58a7c880-431e-4674-94db-821a970ffd52\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a6b11e-vck2d"
Sep 15 20:41:20.259333 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:20.259309    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58a7c880-431e-4674-94db-821a970ffd52-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-a6b11e-vck2d\" (UID: \"58a7c880-431e-4674-94db-821a970ffd52\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a6b11e-vck2d"
Sep 15 20:41:20.314654 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:20.314619    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a6b11e-vck2d"
Sep 15 20:41:20.582533 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:20.582475    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a6b11e-vck2d"]
Sep 15 20:41:20.584836 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:41:20.584806    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58a7c880_431e_4674_94db_821a970ffd52.slice/crio-8fc76659860770135d0a64dbf5f5b491481254e4ca612a6c8ce71a795db4bde4 WatchSource:0}: Error finding container 8fc76659860770135d0a64dbf5f5b491481254e4ca612a6c8ce71a795db4bde4: Status 404 returned error can't find the container with id 8fc76659860770135d0a64dbf5f5b491481254e4ca612a6c8ce71a795db4bde4
Sep 15 20:41:20.651769 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:20.651742    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:20.701201 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:20.701175    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a6b11e-vck2d" event={"ID":"58a7c880-431e-4674-94db-821a970ffd52","Type":"ContainerStarted","Data":"8fc76659860770135d0a64dbf5f5b491481254e4ca612a6c8ce71a795db4bde4"}
Sep 15 20:41:21.360181 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:21.360154    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:21.713449 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:21.713414    2576 generic.go:356] "Generic (PLEG): container finished" podID="58a7c880-431e-4674-94db-821a970ffd52" containerID="57bd0411f3e2f57474d6bc99860667f5bedd1a111c2a38de2cf5609966970d55" exitCode=0
Sep 15 20:41:21.713621 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:21.713497    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a6b11e-vck2d" event={"ID":"58a7c880-431e-4674-94db-821a970ffd52","Type":"ContainerDied","Data":"57bd0411f3e2f57474d6bc99860667f5bedd1a111c2a38de2cf5609966970d55"}
Sep 15 20:41:21.968586 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:21.968469    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:22.673339 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:22.673288    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:22.850375 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:22.850353    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a6b11e-vck2d"
Sep 15 20:41:22.981968 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:22.981930    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58a7c880-431e-4674-94db-821a970ffd52-maas-api-service-ca\") pod \"58a7c880-431e-4674-94db-821a970ffd52\" (UID: \"58a7c880-431e-4674-94db-821a970ffd52\") "
Sep 15 20:41:22.982261 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:22.982237    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58a7c880-431e-4674-94db-821a970ffd52-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "58a7c880-431e-4674-94db-821a970ffd52" (UID: "58a7c880-431e-4674-94db-821a970ffd52"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:41:23.083017 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:23.082982    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58a7c880-431e-4674-94db-821a970ffd52-maas-api-service-ca\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:41:23.280889 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:23.280777    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:23.725427 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:23.725400    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a6b11e-vck2d"
Sep 15 20:41:23.725798 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:23.725403    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a6b11e-vck2d" event={"ID":"58a7c880-431e-4674-94db-821a970ffd52","Type":"ContainerDied","Data":"8fc76659860770135d0a64dbf5f5b491481254e4ca612a6c8ce71a795db4bde4"}
Sep 15 20:41:23.725798 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:23.725519    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8fc76659860770135d0a64dbf5f5b491481254e4ca612a6c8ce71a795db4bde4"
Sep 15 20:41:24.059315 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:24.059211    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:25.659542 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:25.659475    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:26.182677 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:26.182648    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:26.222978 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:26.222948    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a6b11e-5487b545f8-xfd64"]
Sep 15 20:41:26.223202 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:26.223173    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a6b11e-5487b545f8-xfd64" podUID="3e7e1ea1-9075-48bf-b45e-1d8904b40785" containerName="payload-pre-processing" containerID="cri-o://64e660f78aca71449be602dd92dfe4485b896daa1111c250d24e4045b00c271c" gracePeriod=30
Sep 15 20:41:26.252160 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:26.252116    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:26.320351 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:26.320300    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:26.380423 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:26.380391    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a6b11e-vck2d"]
Sep 15 20:41:26.383087 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:26.383058    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:26.383552 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:26.383531    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a6b11e-vck2d"]
Sep 15 20:41:26.440883 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:26.440810    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:26.475128 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:26.475107    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a6b11e-5487b545f8-xfd64"
Sep 15 20:41:26.614274 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:26.614238    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e7e1ea1-9075-48bf-b45e-1d8904b40785-plugins-config-volume\") pod \"3e7e1ea1-9075-48bf-b45e-1d8904b40785\" (UID: \"3e7e1ea1-9075-48bf-b45e-1d8904b40785\") "
Sep 15 20:41:26.614274 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:26.614285    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hkpnm\" (UniqueName: \"kubernetes.io/projected/3e7e1ea1-9075-48bf-b45e-1d8904b40785-kube-api-access-hkpnm\") pod \"3e7e1ea1-9075-48bf-b45e-1d8904b40785\" (UID: \"3e7e1ea1-9075-48bf-b45e-1d8904b40785\") "
Sep 15 20:41:26.614628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:26.614606    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e7e1ea1-9075-48bf-b45e-1d8904b40785-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3e7e1ea1-9075-48bf-b45e-1d8904b40785" (UID: "3e7e1ea1-9075-48bf-b45e-1d8904b40785"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:41:26.616346 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:26.616321    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e7e1ea1-9075-48bf-b45e-1d8904b40785-kube-api-access-hkpnm" (OuterVolumeSpecName: "kube-api-access-hkpnm") pod "3e7e1ea1-9075-48bf-b45e-1d8904b40785" (UID: "3e7e1ea1-9075-48bf-b45e-1d8904b40785"). InnerVolumeSpecName "kube-api-access-hkpnm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:41:26.715965 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:26.715943    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e7e1ea1-9075-48bf-b45e-1d8904b40785-plugins-config-volume\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:41:26.715965 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:26.715966    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkpnm\" (UniqueName: \"kubernetes.io/projected/3e7e1ea1-9075-48bf-b45e-1d8904b40785-kube-api-access-hkpnm\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:41:26.740297 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:26.740272    2576 generic.go:356] "Generic (PLEG): container finished" podID="3e7e1ea1-9075-48bf-b45e-1d8904b40785" containerID="64e660f78aca71449be602dd92dfe4485b896daa1111c250d24e4045b00c271c" exitCode=0
Sep 15 20:41:26.740401 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:26.740316    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a6b11e-5487b545f8-xfd64"
Sep 15 20:41:26.740401 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:26.740324    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a6b11e-5487b545f8-xfd64" event={"ID":"3e7e1ea1-9075-48bf-b45e-1d8904b40785","Type":"ContainerDied","Data":"64e660f78aca71449be602dd92dfe4485b896daa1111c250d24e4045b00c271c"}
Sep 15 20:41:26.740401 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:26.740368    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a6b11e-5487b545f8-xfd64" event={"ID":"3e7e1ea1-9075-48bf-b45e-1d8904b40785","Type":"ContainerDied","Data":"c3dead851ab00eff2053843183b0c8c527fcbab585d68e7c590a10f9d9df93b0"}
Sep 15 20:41:26.740401 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:26.740387    2576 scope.go:117] "RemoveContainer" containerID="64e660f78aca71449be602dd92dfe4485b896daa1111c250d24e4045b00c271c"
Sep 15 20:41:26.751592 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:26.751562    2576 scope.go:117] "RemoveContainer" containerID="64e660f78aca71449be602dd92dfe4485b896daa1111c250d24e4045b00c271c"
Sep 15 20:41:26.752020 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:41:26.751995    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64e660f78aca71449be602dd92dfe4485b896daa1111c250d24e4045b00c271c\": container with ID starting with 64e660f78aca71449be602dd92dfe4485b896daa1111c250d24e4045b00c271c not found: ID does not exist" containerID="64e660f78aca71449be602dd92dfe4485b896daa1111c250d24e4045b00c271c"
Sep 15 20:41:26.752096 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:26.752032    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64e660f78aca71449be602dd92dfe4485b896daa1111c250d24e4045b00c271c"} err="failed to get container status \"64e660f78aca71449be602dd92dfe4485b896daa1111c250d24e4045b00c271c\": rpc error: code = NotFound desc = could not find container \"64e660f78aca71449be602dd92dfe4485b896daa1111c250d24e4045b00c271c\": container with ID starting with 64e660f78aca71449be602dd92dfe4485b896daa1111c250d24e4045b00c271c not found: ID does not exist"
Sep 15 20:41:26.760898 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:26.760842    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:26.761690 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:26.761662    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a6b11e-5487b545f8-xfd64"]
Sep 15 20:41:26.767633 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:26.767607    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a6b11e-5487b545f8-xfd64"]
Sep 15 20:41:26.861905 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:26.861872    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:27.479215 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:27.479175    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:28.144224 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:28.144190    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e7e1ea1-9075-48bf-b45e-1d8904b40785" path="/var/lib/kubelet/pods/3e7e1ea1-9075-48bf-b45e-1d8904b40785/volumes"
Sep 15 20:41:28.144587 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:28.144519    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58a7c880-431e-4674-94db-821a970ffd52" path="/var/lib/kubelet/pods/58a7c880-431e-4674-94db-821a970ffd52/volumes"
Sep 15 20:41:28.400994 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:28.400885    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:28.983715 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:28.983681    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:29.828680 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:29.828645    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:32.527966 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:32.527933    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:32.629723 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:32.629693    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:32.804697 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:32.804620    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:36.465489 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:36.465454    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c005f8-npz4p"]
Sep 15 20:41:36.465889 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:36.465874    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58a7c880-431e-4674-94db-821a970ffd52" containerName="revoke-keys"
Sep 15 20:41:36.465955 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:36.465891    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="58a7c880-431e-4674-94db-821a970ffd52" containerName="revoke-keys"
Sep 15 20:41:36.465955 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:36.465900    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e7e1ea1-9075-48bf-b45e-1d8904b40785" containerName="payload-pre-processing"
Sep 15 20:41:36.465955 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:36.465905    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e7e1ea1-9075-48bf-b45e-1d8904b40785" containerName="payload-pre-processing"
Sep 15 20:41:36.466063 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:36.466017    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="58a7c880-431e-4674-94db-821a970ffd52" containerName="revoke-keys"
Sep 15 20:41:36.466063 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:36.466026    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e7e1ea1-9075-48bf-b45e-1d8904b40785" containerName="payload-pre-processing"
Sep 15 20:41:36.470272 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:36.470254    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c005f8-npz4p"
Sep 15 20:41:36.472391 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:36.472366    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 20:41:36.472528 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:36.472366    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r5lpt\""
Sep 15 20:41:36.476046 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:36.476022    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c005f8-npz4p"]
Sep 15 20:41:36.502471 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:36.502441    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd766f17-46c9-47ed-9033-f6ecb0b79b74-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-c005f8-npz4p\" (UID: \"bd766f17-46c9-47ed-9033-f6ecb0b79b74\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c005f8-npz4p"
Sep 15 20:41:36.603439 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:36.603414    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd766f17-46c9-47ed-9033-f6ecb0b79b74-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-c005f8-npz4p\" (UID: \"bd766f17-46c9-47ed-9033-f6ecb0b79b74\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c005f8-npz4p"
Sep 15 20:41:36.604031 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:36.604014    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd766f17-46c9-47ed-9033-f6ecb0b79b74-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-c005f8-npz4p\" (UID: \"bd766f17-46c9-47ed-9033-f6ecb0b79b74\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c005f8-npz4p"
Sep 15 20:41:36.617581 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:36.617558    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:36.783286 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:36.783198    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c005f8-npz4p"
Sep 15 20:41:37.050033 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:37.050007    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c005f8-npz4p"]
Sep 15 20:41:37.052447 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:41:37.052419    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd766f17_46c9_47ed_9033_f6ecb0b79b74.slice/crio-e14c1570acba7c58cc2a80c5ad15c79e867fbbb650da98a7512e1be230abd01b WatchSource:0}: Error finding container e14c1570acba7c58cc2a80c5ad15c79e867fbbb650da98a7512e1be230abd01b: Status 404 returned error can't find the container with id e14c1570acba7c58cc2a80c5ad15c79e867fbbb650da98a7512e1be230abd01b
Sep 15 20:41:37.479086 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:37.479056    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:37.799580 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:37.799545    2576 generic.go:356] "Generic (PLEG): container finished" podID="bd766f17-46c9-47ed-9033-f6ecb0b79b74" containerID="74543fafff13ec86d40046325f37acbb1fa648016d06a1b42f53aeabf6fa75eb" exitCode=0
Sep 15 20:41:37.799747 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:37.799622    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c005f8-npz4p" event={"ID":"bd766f17-46c9-47ed-9033-f6ecb0b79b74","Type":"ContainerDied","Data":"74543fafff13ec86d40046325f37acbb1fa648016d06a1b42f53aeabf6fa75eb"}
Sep 15 20:41:37.799747 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:37.799677    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c005f8-npz4p" event={"ID":"bd766f17-46c9-47ed-9033-f6ecb0b79b74","Type":"ContainerStarted","Data":"e14c1570acba7c58cc2a80c5ad15c79e867fbbb650da98a7512e1be230abd01b"}
Sep 15 20:41:38.130667 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:38.130631    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:38.862082 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:38.862047    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:38.938068 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:38.938043    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c005f8-npz4p"
Sep 15 20:41:39.024417 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:39.024391    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd766f17-46c9-47ed-9033-f6ecb0b79b74-maas-api-service-ca\") pod \"bd766f17-46c9-47ed-9033-f6ecb0b79b74\" (UID: \"bd766f17-46c9-47ed-9033-f6ecb0b79b74\") "
Sep 15 20:41:39.024725 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:39.024704    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd766f17-46c9-47ed-9033-f6ecb0b79b74-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bd766f17-46c9-47ed-9033-f6ecb0b79b74" (UID: "bd766f17-46c9-47ed-9033-f6ecb0b79b74"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:41:39.125352 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:39.125280    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd766f17-46c9-47ed-9033-f6ecb0b79b74-maas-api-service-ca\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:41:39.460953 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:39.460913    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:39.811482 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:39.811405    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c005f8-npz4p"
Sep 15 20:41:39.811482 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:39.811442    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c005f8-npz4p" event={"ID":"bd766f17-46c9-47ed-9033-f6ecb0b79b74","Type":"ContainerDied","Data":"e14c1570acba7c58cc2a80c5ad15c79e867fbbb650da98a7512e1be230abd01b"}
Sep 15 20:41:39.811678 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:39.811486    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e14c1570acba7c58cc2a80c5ad15c79e867fbbb650da98a7512e1be230abd01b"
Sep 15 20:41:40.258431 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:40.258398    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:51.968883 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:51.968850    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:51.996794 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:51.996759    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c005f8-6ff49f8558-sst7q"]
Sep 15 20:41:51.998129 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:51.997895    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c005f8-6ff49f8558-sst7q" podUID="fbc7e8b8-7c60-4918-95b2-18acc06e2edb" containerName="payload-pre-processing" containerID="cri-o://bc68d9d5f984f8b070d69de5ce56b809348a3185dfc729b06e01ec8f91ee88b3" gracePeriod=30
Sep 15 20:41:52.030026 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:52.030002    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:52.132566 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:52.132538    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:52.151886 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:52.151852    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c005f8-npz4p"]
Sep 15 20:41:52.152102 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:52.152085    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c005f8-npz4p"]
Sep 15 20:41:52.222121 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:52.222023    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:52.244249 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:52.244231    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c005f8-6ff49f8558-sst7q"
Sep 15 20:41:52.340385 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:52.340354    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ldvl6\" (UniqueName: \"kubernetes.io/projected/fbc7e8b8-7c60-4918-95b2-18acc06e2edb-kube-api-access-ldvl6\") pod \"fbc7e8b8-7c60-4918-95b2-18acc06e2edb\" (UID: \"fbc7e8b8-7c60-4918-95b2-18acc06e2edb\") "
Sep 15 20:41:52.340542 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:52.340415    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbc7e8b8-7c60-4918-95b2-18acc06e2edb-plugins-config-volume\") pod \"fbc7e8b8-7c60-4918-95b2-18acc06e2edb\" (UID: \"fbc7e8b8-7c60-4918-95b2-18acc06e2edb\") "
Sep 15 20:41:52.340751 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:52.340728    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbc7e8b8-7c60-4918-95b2-18acc06e2edb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fbc7e8b8-7c60-4918-95b2-18acc06e2edb" (UID: "fbc7e8b8-7c60-4918-95b2-18acc06e2edb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:41:52.342358 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:52.342334    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbc7e8b8-7c60-4918-95b2-18acc06e2edb-kube-api-access-ldvl6" (OuterVolumeSpecName: "kube-api-access-ldvl6") pod "fbc7e8b8-7c60-4918-95b2-18acc06e2edb" (UID: "fbc7e8b8-7c60-4918-95b2-18acc06e2edb"). InnerVolumeSpecName "kube-api-access-ldvl6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:41:52.441815 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:52.441780    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldvl6\" (UniqueName: \"kubernetes.io/projected/fbc7e8b8-7c60-4918-95b2-18acc06e2edb-kube-api-access-ldvl6\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:41:52.441815 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:52.441819    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbc7e8b8-7c60-4918-95b2-18acc06e2edb-plugins-config-volume\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:41:52.515909 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:52.515836    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:52.657861 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:52.657834    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:52.882586 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:52.882507    2576 generic.go:356] "Generic (PLEG): container finished" podID="fbc7e8b8-7c60-4918-95b2-18acc06e2edb" containerID="bc68d9d5f984f8b070d69de5ce56b809348a3185dfc729b06e01ec8f91ee88b3" exitCode=0
Sep 15 20:41:52.882586 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:52.882553    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c005f8-6ff49f8558-sst7q"
Sep 15 20:41:52.882756 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:52.882592    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c005f8-6ff49f8558-sst7q" event={"ID":"fbc7e8b8-7c60-4918-95b2-18acc06e2edb","Type":"ContainerDied","Data":"bc68d9d5f984f8b070d69de5ce56b809348a3185dfc729b06e01ec8f91ee88b3"}
Sep 15 20:41:52.882756 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:52.882648    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c005f8-6ff49f8558-sst7q" event={"ID":"fbc7e8b8-7c60-4918-95b2-18acc06e2edb","Type":"ContainerDied","Data":"7236618cfc4b602f5d5a92756dcea28deef87773d8c85692ce580cdbdae2ef80"}
Sep 15 20:41:52.882756 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:52.882674    2576 scope.go:117] "RemoveContainer" containerID="bc68d9d5f984f8b070d69de5ce56b809348a3185dfc729b06e01ec8f91ee88b3"
Sep 15 20:41:52.891549 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:52.891523    2576 scope.go:117] "RemoveContainer" containerID="bc68d9d5f984f8b070d69de5ce56b809348a3185dfc729b06e01ec8f91ee88b3"
Sep 15 20:41:52.891835 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:41:52.891816    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc68d9d5f984f8b070d69de5ce56b809348a3185dfc729b06e01ec8f91ee88b3\": container with ID starting with bc68d9d5f984f8b070d69de5ce56b809348a3185dfc729b06e01ec8f91ee88b3 not found: ID does not exist" containerID="bc68d9d5f984f8b070d69de5ce56b809348a3185dfc729b06e01ec8f91ee88b3"
Sep 15 20:41:52.891976 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:52.891843    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc68d9d5f984f8b070d69de5ce56b809348a3185dfc729b06e01ec8f91ee88b3"} err="failed to get container status \"bc68d9d5f984f8b070d69de5ce56b809348a3185dfc729b06e01ec8f91ee88b3\": rpc error: code = NotFound desc = could not find container \"bc68d9d5f984f8b070d69de5ce56b809348a3185dfc729b06e01ec8f91ee88b3\": container with ID starting with bc68d9d5f984f8b070d69de5ce56b809348a3185dfc729b06e01ec8f91ee88b3 not found: ID does not exist"
Sep 15 20:41:52.905736 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:52.905713    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c005f8-6ff49f8558-sst7q"]
Sep 15 20:41:52.910054 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:52.910033    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c005f8-6ff49f8558-sst7q"]
Sep 15 20:41:53.337471 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:53.337439    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:54.143547 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:54.143511    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd766f17-46c9-47ed-9033-f6ecb0b79b74" path="/var/lib/kubelet/pods/bd766f17-46c9-47ed-9033-f6ecb0b79b74/volumes"
Sep 15 20:41:54.143913 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:54.143898    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbc7e8b8-7c60-4918-95b2-18acc06e2edb" path="/var/lib/kubelet/pods/fbc7e8b8-7c60-4918-95b2-18acc06e2edb/volumes"
Sep 15 20:41:54.237939 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:54.237896    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:54.963559 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:54.963513    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:41:57.498113 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:57.498075    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v"]
Sep 15 20:41:57.592744 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:41:57.592712    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:42:12.517549 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:12.517448    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v" podUID="a1a79b2d-fbf2-46f5-97a5-3a7689069f89" containerName="main" containerID="cri-o://155effbbaa9d0df80dda47bc88fede8422edd99de732a5cf9594b43c66f9833a" gracePeriod=45
Sep 15 20:42:12.766733 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:12.766710    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v"
Sep 15 20:42:12.926879 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:12.926806    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-tmp-dir\") pod \"a1a79b2d-fbf2-46f5-97a5-3a7689069f89\" (UID: \"a1a79b2d-fbf2-46f5-97a5-3a7689069f89\") "
Sep 15 20:42:12.926879 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:12.926855    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-dshm\") pod \"a1a79b2d-fbf2-46f5-97a5-3a7689069f89\" (UID: \"a1a79b2d-fbf2-46f5-97a5-3a7689069f89\") "
Sep 15 20:42:12.927071 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:12.926898    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-home\") pod \"a1a79b2d-fbf2-46f5-97a5-3a7689069f89\" (UID: \"a1a79b2d-fbf2-46f5-97a5-3a7689069f89\") "
Sep 15 20:42:12.927071 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:12.926969    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-model-cache\") pod \"a1a79b2d-fbf2-46f5-97a5-3a7689069f89\" (UID: \"a1a79b2d-fbf2-46f5-97a5-3a7689069f89\") "
Sep 15 20:42:12.927071 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:12.927038    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-tls-certs\") pod \"a1a79b2d-fbf2-46f5-97a5-3a7689069f89\" (UID: \"a1a79b2d-fbf2-46f5-97a5-3a7689069f89\") "
Sep 15 20:42:12.927237 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:12.927064    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a1a79b2d-fbf2-46f5-97a5-3a7689069f89" (UID: "a1a79b2d-fbf2-46f5-97a5-3a7689069f89"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:42:12.927237 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:12.927080    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvfvc\" (UniqueName: \"kubernetes.io/projected/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-kube-api-access-bvfvc\") pod \"a1a79b2d-fbf2-46f5-97a5-3a7689069f89\" (UID: \"a1a79b2d-fbf2-46f5-97a5-3a7689069f89\") "
Sep 15 20:42:12.927237 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:12.927185    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-home" (OuterVolumeSpecName: "home") pod "a1a79b2d-fbf2-46f5-97a5-3a7689069f89" (UID: "a1a79b2d-fbf2-46f5-97a5-3a7689069f89"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:42:12.927393 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:12.927248    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-model-cache" (OuterVolumeSpecName: "model-cache") pod "a1a79b2d-fbf2-46f5-97a5-3a7689069f89" (UID: "a1a79b2d-fbf2-46f5-97a5-3a7689069f89"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:42:12.927393 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:12.927364    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-tmp-dir\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:42:12.927393 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:12.927378    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-home\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:42:12.927393 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:12.927387    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-model-cache\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:42:12.929129 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:12.929104    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a1a79b2d-fbf2-46f5-97a5-3a7689069f89" (UID: "a1a79b2d-fbf2-46f5-97a5-3a7689069f89"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:42:12.929242 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:12.929128    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-dshm" (OuterVolumeSpecName: "dshm") pod "a1a79b2d-fbf2-46f5-97a5-3a7689069f89" (UID: "a1a79b2d-fbf2-46f5-97a5-3a7689069f89"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:42:12.929242 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:12.929155    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-kube-api-access-bvfvc" (OuterVolumeSpecName: "kube-api-access-bvfvc") pod "a1a79b2d-fbf2-46f5-97a5-3a7689069f89" (UID: "a1a79b2d-fbf2-46f5-97a5-3a7689069f89"). InnerVolumeSpecName "kube-api-access-bvfvc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:42:12.986829 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:12.986796    2576 generic.go:356] "Generic (PLEG): container finished" podID="a1a79b2d-fbf2-46f5-97a5-3a7689069f89" containerID="155effbbaa9d0df80dda47bc88fede8422edd99de732a5cf9594b43c66f9833a" exitCode=0
Sep 15 20:42:12.986951 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:12.986836    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v" event={"ID":"a1a79b2d-fbf2-46f5-97a5-3a7689069f89","Type":"ContainerDied","Data":"155effbbaa9d0df80dda47bc88fede8422edd99de732a5cf9594b43c66f9833a"}
Sep 15 20:42:12.986951 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:12.986877    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v" event={"ID":"a1a79b2d-fbf2-46f5-97a5-3a7689069f89","Type":"ContainerDied","Data":"c03d57ef192d6a61065c5491c6f0760620188e4b64142111c5d7a4d794966064"}
Sep 15 20:42:12.986951 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:12.986893    2576 scope.go:117] "RemoveContainer" containerID="155effbbaa9d0df80dda47bc88fede8422edd99de732a5cf9594b43c66f9833a"
Sep 15 20:42:12.986951 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:12.986845    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v"
Sep 15 20:42:12.995550 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:12.995532    2576 scope.go:117] "RemoveContainer" containerID="155effbbaa9d0df80dda47bc88fede8422edd99de732a5cf9594b43c66f9833a"
Sep 15 20:42:12.995821 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:42:12.995801    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"155effbbaa9d0df80dda47bc88fede8422edd99de732a5cf9594b43c66f9833a\": container with ID starting with 155effbbaa9d0df80dda47bc88fede8422edd99de732a5cf9594b43c66f9833a not found: ID does not exist" containerID="155effbbaa9d0df80dda47bc88fede8422edd99de732a5cf9594b43c66f9833a"
Sep 15 20:42:12.995872 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:12.995829    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"155effbbaa9d0df80dda47bc88fede8422edd99de732a5cf9594b43c66f9833a"} err="failed to get container status \"155effbbaa9d0df80dda47bc88fede8422edd99de732a5cf9594b43c66f9833a\": rpc error: code = NotFound desc = could not find container \"155effbbaa9d0df80dda47bc88fede8422edd99de732a5cf9594b43c66f9833a\": container with ID starting with 155effbbaa9d0df80dda47bc88fede8422edd99de732a5cf9594b43c66f9833a not found: ID does not exist"
Sep 15 20:42:13.006508 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:13.006486    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v"]
Sep 15 20:42:13.010170 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:13.010148    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-c005f8/ipp-route-c005f8-kserve-549d77d459-tjk8v"]
Sep 15 20:42:13.028592 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:13.028573    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-tls-certs\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:42:13.028680 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:13.028596    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvfvc\" (UniqueName: \"kubernetes.io/projected/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-kube-api-access-bvfvc\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:42:13.028680 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:13.028610    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1a79b2d-fbf2-46f5-97a5-3a7689069f89-dshm\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:42:14.143345 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:14.143310    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1a79b2d-fbf2-46f5-97a5-3a7689069f89" path="/var/lib/kubelet/pods/a1a79b2d-fbf2-46f5-97a5-3a7689069f89/volumes"
Sep 15 20:42:34.472094 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:34.472058    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:42:34.586957 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:34.586896    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:42:38.269656 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:38.269621    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:42:40.149631 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:40.149599    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:42:45.889617 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:45.889585    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:42:45.952178 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:45.952154    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:42:46.154243 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:46.154168    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:42:46.473735 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:46.473704    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8dc819-v4rcq"]
Sep 15 20:42:46.474137 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:46.474125    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1a79b2d-fbf2-46f5-97a5-3a7689069f89" containerName="main"
Sep 15 20:42:46.474191 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:46.474139    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1a79b2d-fbf2-46f5-97a5-3a7689069f89" containerName="main"
Sep 15 20:42:46.474191 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:46.474151    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd766f17-46c9-47ed-9033-f6ecb0b79b74" containerName="revoke-keys"
Sep 15 20:42:46.474191 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:46.474159    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd766f17-46c9-47ed-9033-f6ecb0b79b74" containerName="revoke-keys"
Sep 15 20:42:46.474191 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:46.474173    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbc7e8b8-7c60-4918-95b2-18acc06e2edb" containerName="payload-pre-processing"
Sep 15 20:42:46.474191 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:46.474179    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbc7e8b8-7c60-4918-95b2-18acc06e2edb" containerName="payload-pre-processing"
Sep 15 20:42:46.474342 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:46.474238    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1a79b2d-fbf2-46f5-97a5-3a7689069f89" containerName="main"
Sep 15 20:42:46.474342 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:46.474249    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbc7e8b8-7c60-4918-95b2-18acc06e2edb" containerName="payload-pre-processing"
Sep 15 20:42:46.474342 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:46.474257    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd766f17-46c9-47ed-9033-f6ecb0b79b74" containerName="revoke-keys"
Sep 15 20:42:46.478588 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:46.478570    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8dc819-v4rcq"
Sep 15 20:42:46.481102 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:46.481080    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 20:42:46.481206 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:46.481080    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r5lpt\""
Sep 15 20:42:46.483090 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:46.483071    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8dc819-v4rcq"]
Sep 15 20:42:46.571447 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:46.571403    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:42:46.629801 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:46.629773    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63b5107c-66b6-4f11-8f0d-5b90e45f68a1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-8dc819-v4rcq\" (UID: \"63b5107c-66b6-4f11-8f0d-5b90e45f68a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8dc819-v4rcq"
Sep 15 20:42:46.731171 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:46.731143    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63b5107c-66b6-4f11-8f0d-5b90e45f68a1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-8dc819-v4rcq\" (UID: \"63b5107c-66b6-4f11-8f0d-5b90e45f68a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8dc819-v4rcq"
Sep 15 20:42:46.731778 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:46.731758    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63b5107c-66b6-4f11-8f0d-5b90e45f68a1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-8dc819-v4rcq\" (UID: \"63b5107c-66b6-4f11-8f0d-5b90e45f68a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8dc819-v4rcq"
Sep 15 20:42:46.791011 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:46.790984    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8dc819-v4rcq"
Sep 15 20:42:47.062726 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:47.062697    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8dc819-v4rcq"]
Sep 15 20:42:47.064599 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:42:47.064574    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63b5107c_66b6_4f11_8f0d_5b90e45f68a1.slice/crio-fea18b9b139c051775914e91129bd45cf098ada34881d118443affbe5277b37e WatchSource:0}: Error finding container fea18b9b139c051775914e91129bd45cf098ada34881d118443affbe5277b37e: Status 404 returned error can't find the container with id fea18b9b139c051775914e91129bd45cf098ada34881d118443affbe5277b37e
Sep 15 20:42:47.160507 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:47.160471    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8dc819-v4rcq" event={"ID":"63b5107c-66b6-4f11-8f0d-5b90e45f68a1","Type":"ContainerStarted","Data":"fea18b9b139c051775914e91129bd45cf098ada34881d118443affbe5277b37e"}
Sep 15 20:42:47.359287 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:47.359255    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:42:48.010305 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:48.010271    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:42:48.166081 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:48.166050    2576 generic.go:356] "Generic (PLEG): container finished" podID="63b5107c-66b6-4f11-8f0d-5b90e45f68a1" containerID="ddb5c658a1f961e993ea31e3f4986a3fedba25d067eb118d35e1e80bc61a95d8" exitCode=0
Sep 15 20:42:48.166444 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:48.166100    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8dc819-v4rcq" event={"ID":"63b5107c-66b6-4f11-8f0d-5b90e45f68a1","Type":"ContainerDied","Data":"ddb5c658a1f961e993ea31e3f4986a3fedba25d067eb118d35e1e80bc61a95d8"}
Sep 15 20:42:48.647324 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:48.647288    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:42:49.298996 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:49.298974    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8dc819-v4rcq"
Sep 15 20:42:49.339933 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:49.339886    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:42:49.455418 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:49.455393    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63b5107c-66b6-4f11-8f0d-5b90e45f68a1-maas-api-service-ca\") pod \"63b5107c-66b6-4f11-8f0d-5b90e45f68a1\" (UID: \"63b5107c-66b6-4f11-8f0d-5b90e45f68a1\") "
Sep 15 20:42:49.455729 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:49.455709    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63b5107c-66b6-4f11-8f0d-5b90e45f68a1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "63b5107c-66b6-4f11-8f0d-5b90e45f68a1" (UID: "63b5107c-66b6-4f11-8f0d-5b90e45f68a1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:42:49.556850 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:49.556820    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63b5107c-66b6-4f11-8f0d-5b90e45f68a1-maas-api-service-ca\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:42:50.176917 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:50.176886    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8dc819-v4rcq" event={"ID":"63b5107c-66b6-4f11-8f0d-5b90e45f68a1","Type":"ContainerDied","Data":"fea18b9b139c051775914e91129bd45cf098ada34881d118443affbe5277b37e"}
Sep 15 20:42:50.176917 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:50.176937    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8dc819-v4rcq"
Sep 15 20:42:50.177123 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:50.176961    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fea18b9b139c051775914e91129bd45cf098ada34881d118443affbe5277b37e"
Sep 15 20:42:56.974235 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:56.974203    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:42:57.001548 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:57.001514    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-8dc819-7477658649-kdccd"]
Sep 15 20:42:57.001804 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:57.001757    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-8dc819-7477658649-kdccd" podUID="d996108e-e0b4-4379-959c-f816a6d82211" containerName="payload-processing" containerID="cri-o://d2b5e3b774a4be8f534ee43091f8900e8abdcb557fc4699bd58180761cbdc7c5" gracePeriod=30
Sep 15 20:42:57.008438 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:57.008366    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-8dc819-5fd6986f8b-fsn6l"]
Sep 15 20:42:57.008631 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:57.008580    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8dc819-5fd6986f8b-fsn6l" podUID="fd141af0-4e29-40a2-8856-ea19c689ea5e" containerName="payload-pre-processing" containerID="cri-o://0bec450d3e0bf08d900b303295680adcc513cce69a81d1a390c035a8e8d2392e" gracePeriod=30
Sep 15 20:42:57.048463 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:57.048432    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:42:57.109195 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:57.109169    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:42:57.154001 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:57.153964    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8dc819-v4rcq"]
Sep 15 20:42:57.157178 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:57.157152    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8dc819-v4rcq"]
Sep 15 20:42:57.170832 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:57.170806    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:42:57.216886 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:57.216855    2576 generic.go:356] "Generic (PLEG): container finished" podID="fd141af0-4e29-40a2-8856-ea19c689ea5e" containerID="0bec450d3e0bf08d900b303295680adcc513cce69a81d1a390c035a8e8d2392e" exitCode=0
Sep 15 20:42:57.217032 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:57.216950    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8dc819-5fd6986f8b-fsn6l" event={"ID":"fd141af0-4e29-40a2-8856-ea19c689ea5e","Type":"ContainerDied","Data":"0bec450d3e0bf08d900b303295680adcc513cce69a81d1a390c035a8e8d2392e"}
Sep 15 20:42:57.219771 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:57.219750    2576 generic.go:356] "Generic (PLEG): container finished" podID="d996108e-e0b4-4379-959c-f816a6d82211" containerID="d2b5e3b774a4be8f534ee43091f8900e8abdcb557fc4699bd58180761cbdc7c5" exitCode=0
Sep 15 20:42:57.219876 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:57.219784    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-8dc819-7477658649-kdccd" event={"ID":"d996108e-e0b4-4379-959c-f816a6d82211","Type":"ContainerDied","Data":"d2b5e3b774a4be8f534ee43091f8900e8abdcb557fc4699bd58180761cbdc7c5"}
Sep 15 20:42:57.274073 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:57.274050    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-8dc819-7477658649-kdccd"
Sep 15 20:42:57.307745 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:57.307722    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8dc819-5fd6986f8b-fsn6l"
Sep 15 20:42:57.327020 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:57.326992    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd141af0-4e29-40a2-8856-ea19c689ea5e-plugins-config-volume\") pod \"fd141af0-4e29-40a2-8856-ea19c689ea5e\" (UID: \"fd141af0-4e29-40a2-8856-ea19c689ea5e\") "
Sep 15 20:42:57.327118 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:57.327053    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fpj6k\" (UniqueName: \"kubernetes.io/projected/fd141af0-4e29-40a2-8856-ea19c689ea5e-kube-api-access-fpj6k\") pod \"fd141af0-4e29-40a2-8856-ea19c689ea5e\" (UID: \"fd141af0-4e29-40a2-8856-ea19c689ea5e\") "
Sep 15 20:42:57.327178 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:57.327163    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d996108e-e0b4-4379-959c-f816a6d82211-plugins-config-volume\") pod \"d996108e-e0b4-4379-959c-f816a6d82211\" (UID: \"d996108e-e0b4-4379-959c-f816a6d82211\") "
Sep 15 20:42:57.327228 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:57.327211    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2bg5p\" (UniqueName: \"kubernetes.io/projected/d996108e-e0b4-4379-959c-f816a6d82211-kube-api-access-2bg5p\") pod \"d996108e-e0b4-4379-959c-f816a6d82211\" (UID: \"d996108e-e0b4-4379-959c-f816a6d82211\") "
Sep 15 20:42:57.327447 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:57.327385    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd141af0-4e29-40a2-8856-ea19c689ea5e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fd141af0-4e29-40a2-8856-ea19c689ea5e" (UID: "fd141af0-4e29-40a2-8856-ea19c689ea5e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:42:57.327535 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:57.327496    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd141af0-4e29-40a2-8856-ea19c689ea5e-plugins-config-volume\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:42:57.327820 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:57.327585    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d996108e-e0b4-4379-959c-f816a6d82211-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d996108e-e0b4-4379-959c-f816a6d82211" (UID: "d996108e-e0b4-4379-959c-f816a6d82211"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:42:57.329405 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:57.329379    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d996108e-e0b4-4379-959c-f816a6d82211-kube-api-access-2bg5p" (OuterVolumeSpecName: "kube-api-access-2bg5p") pod "d996108e-e0b4-4379-959c-f816a6d82211" (UID: "d996108e-e0b4-4379-959c-f816a6d82211"). InnerVolumeSpecName "kube-api-access-2bg5p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:42:57.329648 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:57.329622    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd141af0-4e29-40a2-8856-ea19c689ea5e-kube-api-access-fpj6k" (OuterVolumeSpecName: "kube-api-access-fpj6k") pod "fd141af0-4e29-40a2-8856-ea19c689ea5e" (UID: "fd141af0-4e29-40a2-8856-ea19c689ea5e"). InnerVolumeSpecName "kube-api-access-fpj6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:42:57.428538 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:57.428507    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpj6k\" (UniqueName: \"kubernetes.io/projected/fd141af0-4e29-40a2-8856-ea19c689ea5e-kube-api-access-fpj6k\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:42:57.428538 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:57.428532    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d996108e-e0b4-4379-959c-f816a6d82211-plugins-config-volume\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:42:57.428692 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:57.428545    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bg5p\" (UniqueName: \"kubernetes.io/projected/d996108e-e0b4-4379-959c-f816a6d82211-kube-api-access-2bg5p\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:42:57.525238 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:57.525152    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:42:57.626076 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:57.626046    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:42:58.143150 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:58.143120    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63b5107c-66b6-4f11-8f0d-5b90e45f68a1" path="/var/lib/kubelet/pods/63b5107c-66b6-4f11-8f0d-5b90e45f68a1/volumes"
Sep 15 20:42:58.225979 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:58.225950    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8dc819-5fd6986f8b-fsn6l"
Sep 15 20:42:58.226122 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:58.225977    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8dc819-5fd6986f8b-fsn6l" event={"ID":"fd141af0-4e29-40a2-8856-ea19c689ea5e","Type":"ContainerDied","Data":"3297992ce892fac7ee2cff3a396468b3c5e6a95ea3336e9372beeba5230d4e08"}
Sep 15 20:42:58.226122 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:58.226039    2576 scope.go:117] "RemoveContainer" containerID="0bec450d3e0bf08d900b303295680adcc513cce69a81d1a390c035a8e8d2392e"
Sep 15 20:42:58.227537 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:58.227502    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-8dc819-7477658649-kdccd" event={"ID":"d996108e-e0b4-4379-959c-f816a6d82211","Type":"ContainerDied","Data":"fe2e65c94b99531e70cb26c07cae98f48a3dc5573b333b18ec618a2827880666"}
Sep 15 20:42:58.227646 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:58.227583    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-8dc819-7477658649-kdccd"
Sep 15 20:42:58.233050 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:58.233029    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:42:58.235475 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:58.235446    2576 scope.go:117] "RemoveContainer" containerID="d2b5e3b774a4be8f534ee43091f8900e8abdcb557fc4699bd58180761cbdc7c5"
Sep 15 20:42:58.246979 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:58.246959    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-8dc819-7477658649-kdccd"]
Sep 15 20:42:58.251161 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:58.251139    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-8dc819-7477658649-kdccd"]
Sep 15 20:42:58.261233 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:58.261211    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-8dc819-5fd6986f8b-fsn6l"]
Sep 15 20:42:58.265616 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:58.265596    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-8dc819-5fd6986f8b-fsn6l"]
Sep 15 20:42:58.868676 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:42:58.868640    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:43:00.144386 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:00.144355    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d996108e-e0b4-4379-959c-f816a6d82211" path="/var/lib/kubelet/pods/d996108e-e0b4-4379-959c-f816a6d82211/volumes"
Sep 15 20:43:00.144759 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:00.144716    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd141af0-4e29-40a2-8856-ea19c689ea5e" path="/var/lib/kubelet/pods/fd141af0-4e29-40a2-8856-ea19c689ea5e/volumes"
Sep 15 20:43:03.370779 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:03.370735    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:43:03.471456 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:03.471418    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:43:03.639867 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:03.639783    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:43:03.975186 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:03.975153    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-92702a-mbxjq"]
Sep 15 20:43:03.975599 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:03.975584    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd141af0-4e29-40a2-8856-ea19c689ea5e" containerName="payload-pre-processing"
Sep 15 20:43:03.975656 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:03.975604    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd141af0-4e29-40a2-8856-ea19c689ea5e" containerName="payload-pre-processing"
Sep 15 20:43:03.975656 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:03.975616    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63b5107c-66b6-4f11-8f0d-5b90e45f68a1" containerName="revoke-keys"
Sep 15 20:43:03.975656 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:03.975621    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="63b5107c-66b6-4f11-8f0d-5b90e45f68a1" containerName="revoke-keys"
Sep 15 20:43:03.975656 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:03.975642    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d996108e-e0b4-4379-959c-f816a6d82211" containerName="payload-processing"
Sep 15 20:43:03.975656 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:03.975648    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d996108e-e0b4-4379-959c-f816a6d82211" containerName="payload-processing"
Sep 15 20:43:03.975805 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:03.975709    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d996108e-e0b4-4379-959c-f816a6d82211" containerName="payload-processing"
Sep 15 20:43:03.975805 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:03.975719    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd141af0-4e29-40a2-8856-ea19c689ea5e" containerName="payload-pre-processing"
Sep 15 20:43:03.975805 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:03.975727    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="63b5107c-66b6-4f11-8f0d-5b90e45f68a1" containerName="revoke-keys"
Sep 15 20:43:03.980319 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:03.980302    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-92702a-mbxjq"
Sep 15 20:43:03.982742 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:03.982713    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r5lpt\""
Sep 15 20:43:03.983068 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:03.983048    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 20:43:03.985614 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:03.985589    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-92702a-mbxjq"]
Sep 15 20:43:04.083795 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:04.083766    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b20c9077-43b8-4730-b70d-7f3173386fab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-92702a-mbxjq\" (UID: \"b20c9077-43b8-4730-b70d-7f3173386fab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-92702a-mbxjq"
Sep 15 20:43:04.184727 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:04.184698    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b20c9077-43b8-4730-b70d-7f3173386fab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-92702a-mbxjq\" (UID: \"b20c9077-43b8-4730-b70d-7f3173386fab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-92702a-mbxjq"
Sep 15 20:43:04.185361 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:04.185340    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b20c9077-43b8-4730-b70d-7f3173386fab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-92702a-mbxjq\" (UID: \"b20c9077-43b8-4730-b70d-7f3173386fab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-92702a-mbxjq"
Sep 15 20:43:04.292201 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:04.292127    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-92702a-mbxjq"
Sep 15 20:43:04.560723 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:04.560691    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-92702a-mbxjq"]
Sep 15 20:43:04.562763 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:43:04.562722    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb20c9077_43b8_4730_b70d_7f3173386fab.slice/crio-1c280b67d4fd611489b3defef1796146cd5282ba7f3b6eff9699c3745262dfcc WatchSource:0}: Error finding container 1c280b67d4fd611489b3defef1796146cd5282ba7f3b6eff9699c3745262dfcc: Status 404 returned error can't find the container with id 1c280b67d4fd611489b3defef1796146cd5282ba7f3b6eff9699c3745262dfcc
Sep 15 20:43:04.858588 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:04.858557    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:43:05.270545 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:05.270516    2576 generic.go:356] "Generic (PLEG): container finished" podID="b20c9077-43b8-4730-b70d-7f3173386fab" containerID="1983e12c4e27976beb04116939a29ef99f986d98d7ebdb6848ad3ff67304d58b" exitCode=0
Sep 15 20:43:05.270664 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:05.270571    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-92702a-mbxjq" event={"ID":"b20c9077-43b8-4730-b70d-7f3173386fab","Type":"ContainerDied","Data":"1983e12c4e27976beb04116939a29ef99f986d98d7ebdb6848ad3ff67304d58b"}
Sep 15 20:43:05.270664 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:05.270624    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-92702a-mbxjq" event={"ID":"b20c9077-43b8-4730-b70d-7f3173386fab","Type":"ContainerStarted","Data":"1c280b67d4fd611489b3defef1796146cd5282ba7f3b6eff9699c3745262dfcc"}
Sep 15 20:43:05.477341 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:05.477308    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:43:06.089248 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:06.089215    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:43:06.398796 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:06.398775    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-92702a-mbxjq"
Sep 15 20:43:06.506097 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:06.506069    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b20c9077-43b8-4730-b70d-7f3173386fab-maas-api-service-ca\") pod \"b20c9077-43b8-4730-b70d-7f3173386fab\" (UID: \"b20c9077-43b8-4730-b70d-7f3173386fab\") "
Sep 15 20:43:06.506406 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:06.506385    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b20c9077-43b8-4730-b70d-7f3173386fab-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b20c9077-43b8-4730-b70d-7f3173386fab" (UID: "b20c9077-43b8-4730-b70d-7f3173386fab"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:43:06.607227 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:06.607200    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b20c9077-43b8-4730-b70d-7f3173386fab-maas-api-service-ca\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:43:07.280856 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:07.280825    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-92702a-mbxjq"
Sep 15 20:43:07.281247 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:07.280876    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-92702a-mbxjq" event={"ID":"b20c9077-43b8-4730-b70d-7f3173386fab","Type":"ContainerDied","Data":"1c280b67d4fd611489b3defef1796146cd5282ba7f3b6eff9699c3745262dfcc"}
Sep 15 20:43:07.281247 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:07.280965    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1c280b67d4fd611489b3defef1796146cd5282ba7f3b6eff9699c3745262dfcc"
Sep 15 20:43:14.430659 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:14.430622    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:43:14.452080 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:14.452041    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-92702a-d74467ff7-6mtgx"]
Sep 15 20:43:14.452327 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:14.452298    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-92702a-d74467ff7-6mtgx" podUID="5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a" containerName="payload-processing" containerID="cri-o://3a4b8b9d90fc9b0e612d8b66c3c304a6a9cfd0994e1cd63e02b76e650e8f6094" gracePeriod=30
Sep 15 20:43:14.457513 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:14.457482    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-92702a-7865d5fc58-zgbpp"]
Sep 15 20:43:14.458259 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:14.457755    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-92702a-7865d5fc58-zgbpp" podUID="d56d4079-5452-4e54-82fb-06cf8a4ce399" containerName="payload-pre-processing" containerID="cri-o://19a5d72aa1af58a22ae4b5993d41961e3b2318137a230d381d627f449139ddea" gracePeriod=30
Sep 15 20:43:14.526296 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:14.526267    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:43:14.602538 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:14.602502    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-92702a-mbxjq"]
Sep 15 20:43:14.603805 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:14.603734    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:43:14.606320 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:14.606269    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-92702a-mbxjq"]
Sep 15 20:43:14.679830 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:14.679797    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:43:14.707064 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:14.707040    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-92702a-d74467ff7-6mtgx"
Sep 15 20:43:14.727322 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:14.727295    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-92702a-7865d5fc58-zgbpp"
Sep 15 20:43:14.777084 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:14.777055    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27976\" (UniqueName: \"kubernetes.io/projected/d56d4079-5452-4e54-82fb-06cf8a4ce399-kube-api-access-27976\") pod \"d56d4079-5452-4e54-82fb-06cf8a4ce399\" (UID: \"d56d4079-5452-4e54-82fb-06cf8a4ce399\") "
Sep 15 20:43:14.777249 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:14.777148    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kd79g\" (UniqueName: \"kubernetes.io/projected/5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a-kube-api-access-kd79g\") pod \"5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a\" (UID: \"5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a\") "
Sep 15 20:43:14.777249 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:14.777182    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a-plugins-config-volume\") pod \"5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a\" (UID: \"5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a\") "
Sep 15 20:43:14.777389 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:14.777350    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d56d4079-5452-4e54-82fb-06cf8a4ce399-plugins-config-volume\") pod \"d56d4079-5452-4e54-82fb-06cf8a4ce399\" (UID: \"d56d4079-5452-4e54-82fb-06cf8a4ce399\") "
Sep 15 20:43:14.777690 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:14.777662    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a" (UID: "5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:43:14.777874 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:14.777690    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d56d4079-5452-4e54-82fb-06cf8a4ce399-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d56d4079-5452-4e54-82fb-06cf8a4ce399" (UID: "d56d4079-5452-4e54-82fb-06cf8a4ce399"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:43:14.779360 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:14.779334    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a-kube-api-access-kd79g" (OuterVolumeSpecName: "kube-api-access-kd79g") pod "5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a" (UID: "5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a"). InnerVolumeSpecName "kube-api-access-kd79g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:43:14.779454 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:14.779382    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d56d4079-5452-4e54-82fb-06cf8a4ce399-kube-api-access-27976" (OuterVolumeSpecName: "kube-api-access-27976") pod "d56d4079-5452-4e54-82fb-06cf8a4ce399" (UID: "d56d4079-5452-4e54-82fb-06cf8a4ce399"). InnerVolumeSpecName "kube-api-access-27976". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:43:14.878365 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:14.878333    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d56d4079-5452-4e54-82fb-06cf8a4ce399-plugins-config-volume\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:43:14.878365 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:14.878358    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27976\" (UniqueName: \"kubernetes.io/projected/d56d4079-5452-4e54-82fb-06cf8a4ce399-kube-api-access-27976\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:43:14.878365 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:14.878368    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kd79g\" (UniqueName: \"kubernetes.io/projected/5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a-kube-api-access-kd79g\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:43:14.878599 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:14.878377    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a-plugins-config-volume\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:43:14.965328 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:14.965280    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:43:15.066517 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:15.066484    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:43:15.320692 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:15.320604    2576 generic.go:356] "Generic (PLEG): container finished" podID="5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a" containerID="3a4b8b9d90fc9b0e612d8b66c3c304a6a9cfd0994e1cd63e02b76e650e8f6094" exitCode=0
Sep 15 20:43:15.320692 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:15.320648    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-92702a-d74467ff7-6mtgx"
Sep 15 20:43:15.320954 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:15.320691    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-92702a-d74467ff7-6mtgx" event={"ID":"5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a","Type":"ContainerDied","Data":"3a4b8b9d90fc9b0e612d8b66c3c304a6a9cfd0994e1cd63e02b76e650e8f6094"}
Sep 15 20:43:15.320954 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:15.320748    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-92702a-d74467ff7-6mtgx" event={"ID":"5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a","Type":"ContainerDied","Data":"5cfb4e26fdc8430153f8f2af0a75ba385b9567481b6d0c4d3bfecdc6eb24c77d"}
Sep 15 20:43:15.320954 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:15.320777    2576 scope.go:117] "RemoveContainer" containerID="3a4b8b9d90fc9b0e612d8b66c3c304a6a9cfd0994e1cd63e02b76e650e8f6094"
Sep 15 20:43:15.322121 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:15.322092    2576 generic.go:356] "Generic (PLEG): container finished" podID="d56d4079-5452-4e54-82fb-06cf8a4ce399" containerID="19a5d72aa1af58a22ae4b5993d41961e3b2318137a230d381d627f449139ddea" exitCode=0
Sep 15 20:43:15.322239 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:15.322121    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-92702a-7865d5fc58-zgbpp" event={"ID":"d56d4079-5452-4e54-82fb-06cf8a4ce399","Type":"ContainerDied","Data":"19a5d72aa1af58a22ae4b5993d41961e3b2318137a230d381d627f449139ddea"}
Sep 15 20:43:15.322239 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:15.322143    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-92702a-7865d5fc58-zgbpp"
Sep 15 20:43:15.322239 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:15.322154    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-92702a-7865d5fc58-zgbpp" event={"ID":"d56d4079-5452-4e54-82fb-06cf8a4ce399","Type":"ContainerDied","Data":"bec99a280350a83c828336a02dd2410759a3e2908d34c4d3cfbda6c4caf8dd71"}
Sep 15 20:43:15.331312 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:15.331297    2576 scope.go:117] "RemoveContainer" containerID="3a4b8b9d90fc9b0e612d8b66c3c304a6a9cfd0994e1cd63e02b76e650e8f6094"
Sep 15 20:43:15.331592 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:43:15.331567    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a4b8b9d90fc9b0e612d8b66c3c304a6a9cfd0994e1cd63e02b76e650e8f6094\": container with ID starting with 3a4b8b9d90fc9b0e612d8b66c3c304a6a9cfd0994e1cd63e02b76e650e8f6094 not found: ID does not exist" containerID="3a4b8b9d90fc9b0e612d8b66c3c304a6a9cfd0994e1cd63e02b76e650e8f6094"
Sep 15 20:43:15.331845 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:15.331601    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a4b8b9d90fc9b0e612d8b66c3c304a6a9cfd0994e1cd63e02b76e650e8f6094"} err="failed to get container status \"3a4b8b9d90fc9b0e612d8b66c3c304a6a9cfd0994e1cd63e02b76e650e8f6094\": rpc error: code = NotFound desc = could not find container \"3a4b8b9d90fc9b0e612d8b66c3c304a6a9cfd0994e1cd63e02b76e650e8f6094\": container with ID starting with 3a4b8b9d90fc9b0e612d8b66c3c304a6a9cfd0994e1cd63e02b76e650e8f6094 not found: ID does not exist"
Sep 15 20:43:15.331845 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:15.331621    2576 scope.go:117] "RemoveContainer" containerID="19a5d72aa1af58a22ae4b5993d41961e3b2318137a230d381d627f449139ddea"
Sep 15 20:43:15.339133 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:15.339114    2576 scope.go:117] "RemoveContainer" containerID="19a5d72aa1af58a22ae4b5993d41961e3b2318137a230d381d627f449139ddea"
Sep 15 20:43:15.339380 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:43:15.339363    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19a5d72aa1af58a22ae4b5993d41961e3b2318137a230d381d627f449139ddea\": container with ID starting with 19a5d72aa1af58a22ae4b5993d41961e3b2318137a230d381d627f449139ddea not found: ID does not exist" containerID="19a5d72aa1af58a22ae4b5993d41961e3b2318137a230d381d627f449139ddea"
Sep 15 20:43:15.339435 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:15.339387    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19a5d72aa1af58a22ae4b5993d41961e3b2318137a230d381d627f449139ddea"} err="failed to get container status \"19a5d72aa1af58a22ae4b5993d41961e3b2318137a230d381d627f449139ddea\": rpc error: code = NotFound desc = could not find container \"19a5d72aa1af58a22ae4b5993d41961e3b2318137a230d381d627f449139ddea\": container with ID starting with 19a5d72aa1af58a22ae4b5993d41961e3b2318137a230d381d627f449139ddea not found: ID does not exist"
Sep 15 20:43:15.342212 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:15.342191    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-92702a-d74467ff7-6mtgx"]
Sep 15 20:43:15.348246 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:15.348224    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-92702a-d74467ff7-6mtgx"]
Sep 15 20:43:15.356630 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:15.356609    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-92702a-7865d5fc58-zgbpp"]
Sep 15 20:43:15.360226 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:15.360206    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-92702a-7865d5fc58-zgbpp"]
Sep 15 20:43:15.784996 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:15.784957    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:43:16.143330 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:16.143253    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a" path="/var/lib/kubelet/pods/5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a/volumes"
Sep 15 20:43:16.143612 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:16.143598    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b20c9077-43b8-4730-b70d-7f3173386fab" path="/var/lib/kubelet/pods/b20c9077-43b8-4730-b70d-7f3173386fab/volumes"
Sep 15 20:43:16.143911 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:16.143900    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d56d4079-5452-4e54-82fb-06cf8a4ce399" path="/var/lib/kubelet/pods/d56d4079-5452-4e54-82fb-06cf8a4ce399/volumes"
Sep 15 20:43:20.795808 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:20.795775    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:43:20.860461 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:20.860430    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:43:21.050306 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:21.050228    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:43:21.367131 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:21.367054    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e80b38-l522k"]
Sep 15 20:43:21.367434 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:21.367416    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b20c9077-43b8-4730-b70d-7f3173386fab" containerName="revoke-keys"
Sep 15 20:43:21.367479 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:21.367436    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b20c9077-43b8-4730-b70d-7f3173386fab" containerName="revoke-keys"
Sep 15 20:43:21.367479 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:21.367451    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d56d4079-5452-4e54-82fb-06cf8a4ce399" containerName="payload-pre-processing"
Sep 15 20:43:21.367479 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:21.367457    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d56d4079-5452-4e54-82fb-06cf8a4ce399" containerName="payload-pre-processing"
Sep 15 20:43:21.367479 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:21.367467    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a" containerName="payload-processing"
Sep 15 20:43:21.367479 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:21.367472    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a" containerName="payload-processing"
Sep 15 20:43:21.367636 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:21.367538    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b20c9077-43b8-4730-b70d-7f3173386fab" containerName="revoke-keys"
Sep 15 20:43:21.367636 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:21.367548    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d56d4079-5452-4e54-82fb-06cf8a4ce399" containerName="payload-pre-processing"
Sep 15 20:43:21.367636 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:21.367554    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5aff0db4-123b-4f5b-af5d-6c8ca96e3f0a" containerName="payload-processing"
Sep 15 20:43:21.371817 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:21.371798    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e80b38-l522k"
Sep 15 20:43:21.374065 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:21.374044    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 20:43:21.374162 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:21.374098    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r5lpt\""
Sep 15 20:43:21.378786 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:21.378761    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e80b38-l522k"]
Sep 15 20:43:21.438837 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:21.438803    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8443905a-9499-462e-8145-1994d7587704-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-e80b38-l522k\" (UID: \"8443905a-9499-462e-8145-1994d7587704\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e80b38-l522k"
Sep 15 20:43:21.540243 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:21.540205    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8443905a-9499-462e-8145-1994d7587704-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-e80b38-l522k\" (UID: \"8443905a-9499-462e-8145-1994d7587704\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e80b38-l522k"
Sep 15 20:43:21.540743 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:21.540722    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8443905a-9499-462e-8145-1994d7587704-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-e80b38-l522k\" (UID: \"8443905a-9499-462e-8145-1994d7587704\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e80b38-l522k"
Sep 15 20:43:21.683516 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:21.683434    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e80b38-l522k"
Sep 15 20:43:21.953548 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:21.953525    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e80b38-l522k"]
Sep 15 20:43:21.959444 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:43:21.959386    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8443905a_9499_462e_8145_1994d7587704.slice/crio-824a0e7034121c93742ae08c44be1de91dbbf1d3da265877f02e3e084d027e1c WatchSource:0}: Error finding container 824a0e7034121c93742ae08c44be1de91dbbf1d3da265877f02e3e084d027e1c: Status 404 returned error can't find the container with id 824a0e7034121c93742ae08c44be1de91dbbf1d3da265877f02e3e084d027e1c
Sep 15 20:43:22.145607 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:22.145573    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:43:22.359812 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:22.359776    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e80b38-l522k" event={"ID":"8443905a-9499-462e-8145-1994d7587704","Type":"ContainerStarted","Data":"6b74f36afc5e633342c830f912a3933611b260ddcdcecb3615844087b61e6141"}
Sep 15 20:43:22.360002 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:22.359828    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e80b38-l522k" event={"ID":"8443905a-9499-462e-8145-1994d7587704","Type":"ContainerStarted","Data":"824a0e7034121c93742ae08c44be1de91dbbf1d3da265877f02e3e084d027e1c"}
Sep 15 20:43:22.374950 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:22.374889    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e80b38-l522k" podStartSLOduration=1.374874847 podStartE2EDuration="1.374874847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:43:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:43:22.373353132 +0000 UTC m=+2316.861817622" watchObservedRunningTime="2026-09-15 20:43:22.374874847 +0000 UTC m=+2316.863339323"
Sep 15 20:43:22.816609 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:22.816576    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:43:23.366012 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:23.365980    2576 generic.go:356] "Generic (PLEG): container finished" podID="8443905a-9499-462e-8145-1994d7587704" containerID="6b74f36afc5e633342c830f912a3933611b260ddcdcecb3615844087b61e6141" exitCode=0
Sep 15 20:43:23.366396 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:23.366043    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e80b38-l522k" event={"ID":"8443905a-9499-462e-8145-1994d7587704","Type":"ContainerDied","Data":"6b74f36afc5e633342c830f912a3933611b260ddcdcecb3615844087b61e6141"}
Sep 15 20:43:24.495638 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:24.495615    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e80b38-l522k"
Sep 15 20:43:24.567674 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:24.567645    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8443905a-9499-462e-8145-1994d7587704-maas-api-service-ca\") pod \"8443905a-9499-462e-8145-1994d7587704\" (UID: \"8443905a-9499-462e-8145-1994d7587704\") "
Sep 15 20:43:24.567989 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:24.567968    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8443905a-9499-462e-8145-1994d7587704-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8443905a-9499-462e-8145-1994d7587704" (UID: "8443905a-9499-462e-8145-1994d7587704"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:43:24.669142 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:24.669071    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8443905a-9499-462e-8145-1994d7587704-maas-api-service-ca\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:43:25.377903 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:25.377874    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e80b38-l522k"
Sep 15 20:43:25.378094 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:25.377939    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e80b38-l522k" event={"ID":"8443905a-9499-462e-8145-1994d7587704","Type":"ContainerDied","Data":"824a0e7034121c93742ae08c44be1de91dbbf1d3da265877f02e3e084d027e1c"}
Sep 15 20:43:25.378094 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:25.378005    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="824a0e7034121c93742ae08c44be1de91dbbf1d3da265877f02e3e084d027e1c"
Sep 15 20:43:31.890193 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:31.890158    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:43:31.957622 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:31.957589    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:43:32.021943 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:32.021885    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:43:32.074099 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:32.074061    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e80b38-l522k"]
Sep 15 20:43:32.078779 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:32.078754    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-e80b38-l522k"]
Sep 15 20:43:32.080735 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:32.080712    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:43:32.143568 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:32.143487    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8443905a-9499-462e-8145-1994d7587704" path="/var/lib/kubelet/pods/8443905a-9499-462e-8145-1994d7587704/volumes"
Sep 15 20:43:32.146421 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:32.146388    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:43:32.452103 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:32.452026    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:43:32.651816 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:32.651783    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:43:34.998853 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:34.998820    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:43:38.481502 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:38.481427    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:43:38.584226 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:38.584196    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:43:38.684813 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:38.684781    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:43:38.884613 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:43:38.884541    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:44:25.693575 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:44:25.693537    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:44:25.747599 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:44:25.747570    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:44:32.609216 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:44:32.609180    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:45:00.135791 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:45:00.135753    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29825085-tgcdz"]
Sep 15 20:45:00.136277 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:45:00.136136    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8443905a-9499-462e-8145-1994d7587704" containerName="revoke-keys"
Sep 15 20:45:00.136277 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:45:00.136149    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8443905a-9499-462e-8145-1994d7587704" containerName="revoke-keys"
Sep 15 20:45:00.136277 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:45:00.136213    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8443905a-9499-462e-8145-1994d7587704" containerName="revoke-keys"
Sep 15 20:45:00.139229 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:45:00.139209    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29825085-tgcdz"
Sep 15 20:45:00.141293 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:45:00.141270    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r5lpt\""
Sep 15 20:45:00.146446 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:45:00.146421    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29825085-tgcdz"]
Sep 15 20:45:00.150455 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:45:00.150434    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29825085-tgcdz"
Sep 15 20:45:00.418050 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:45:00.418026    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29825085-tgcdz"]
Sep 15 20:45:00.420367 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:45:00.420341    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72939fb6_0f65_43cc_b956_e212029fbd27.slice/crio-25e9eabbd2fa5f7d7880bc6efe2ed1da114d40032390b4d53f9de955b0f29c58 WatchSource:0}: Error finding container 25e9eabbd2fa5f7d7880bc6efe2ed1da114d40032390b4d53f9de955b0f29c58: Status 404 returned error can't find the container with id 25e9eabbd2fa5f7d7880bc6efe2ed1da114d40032390b4d53f9de955b0f29c58
Sep 15 20:45:00.850636 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:45:00.850595    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29825085-tgcdz" event={"ID":"72939fb6-0f65-43cc-b956-e212029fbd27","Type":"ContainerStarted","Data":"05d279091eac9008570e0ddba5d585bd29d88a2d482d6beb1b96764fd51701a6"}
Sep 15 20:45:00.850794 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:45:00.850652    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29825085-tgcdz" event={"ID":"72939fb6-0f65-43cc-b956-e212029fbd27","Type":"ContainerStarted","Data":"25e9eabbd2fa5f7d7880bc6efe2ed1da114d40032390b4d53f9de955b0f29c58"}
Sep 15 20:45:00.865422 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:45:00.865374    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29825085-tgcdz" podStartSLOduration=0.865359538 podStartE2EDuration="865.359538ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:45:00.863342008 +0000 UTC m=+2415.351806485" watchObservedRunningTime="2026-09-15 20:45:00.865359538 +0000 UTC m=+2415.353824013"
Sep 15 20:45:01.857772 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:45:01.857732    2576 generic.go:356] "Generic (PLEG): container finished" podID="72939fb6-0f65-43cc-b956-e212029fbd27" containerID="05d279091eac9008570e0ddba5d585bd29d88a2d482d6beb1b96764fd51701a6" exitCode=0
Sep 15 20:45:01.858247 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:45:01.857813    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29825085-tgcdz" event={"ID":"72939fb6-0f65-43cc-b956-e212029fbd27","Type":"ContainerDied","Data":"05d279091eac9008570e0ddba5d585bd29d88a2d482d6beb1b96764fd51701a6"}
Sep 15 20:45:02.997428 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:45:02.997405    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29825085-tgcdz"
Sep 15 20:45:03.868897 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:45:03.868862    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29825085-tgcdz" event={"ID":"72939fb6-0f65-43cc-b956-e212029fbd27","Type":"ContainerDied","Data":"25e9eabbd2fa5f7d7880bc6efe2ed1da114d40032390b4d53f9de955b0f29c58"}
Sep 15 20:45:03.869098 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:45:03.868912    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="25e9eabbd2fa5f7d7880bc6efe2ed1da114d40032390b4d53f9de955b0f29c58"
Sep 15 20:45:03.869098 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:45:03.868869    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29825085-tgcdz"
Sep 15 20:45:05.300073 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:45:05.300037    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:45:06.674227 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:45:06.674196    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:47:21.499673 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:47:21.499639    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:47:46.486864 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:47:46.486831    2576 scope.go:117] "RemoveContainer" containerID="57bd0411f3e2f57474d6bc99860667f5bedd1a111c2a38de2cf5609966970d55"
Sep 15 20:47:46.495237 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:47:46.495220    2576 scope.go:117] "RemoveContainer" containerID="74543fafff13ec86d40046325f37acbb1fa648016d06a1b42f53aeabf6fa75eb"
Sep 15 20:48:05.033785 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:05.033753    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp"]
Sep 15 20:48:05.034339 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:05.034318    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72939fb6-0f65-43cc-b956-e212029fbd27" containerName="cleanup"
Sep 15 20:48:05.034418 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:05.034343    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="72939fb6-0f65-43cc-b956-e212029fbd27" containerName="cleanup"
Sep 15 20:48:05.034476 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:05.034454    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="72939fb6-0f65-43cc-b956-e212029fbd27" containerName="cleanup"
Sep 15 20:48:05.037437 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:05.037417    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp"
Sep 15 20:48:05.039756 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:05.039734    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-98d6415a-kserve-self-signed-certs\""
Sep 15 20:48:05.047622 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:05.047599    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp"]
Sep 15 20:48:05.072628 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:05.072601    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:48:05.086801 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:05.086779    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f2deabb-1eac-4e09-a371-5b14044bc223-tmp-dir\") pod \"e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp\" (UID: \"6f2deabb-1eac-4e09-a371-5b14044bc223\") " pod="llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp"
Sep 15 20:48:05.086878 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:05.086813    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjlpv\" (UniqueName: \"kubernetes.io/projected/6f2deabb-1eac-4e09-a371-5b14044bc223-kube-api-access-vjlpv\") pod \"e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp\" (UID: \"6f2deabb-1eac-4e09-a371-5b14044bc223\") " pod="llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp"
Sep 15 20:48:05.086878 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:05.086838    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f2deabb-1eac-4e09-a371-5b14044bc223-dshm\") pod \"e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp\" (UID: \"6f2deabb-1eac-4e09-a371-5b14044bc223\") " pod="llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp"
Sep 15 20:48:05.086987 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:05.086947    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f2deabb-1eac-4e09-a371-5b14044bc223-model-cache\") pod \"e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp\" (UID: \"6f2deabb-1eac-4e09-a371-5b14044bc223\") " pod="llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp"
Sep 15 20:48:05.087020 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:05.087013    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f2deabb-1eac-4e09-a371-5b14044bc223-home\") pod \"e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp\" (UID: \"6f2deabb-1eac-4e09-a371-5b14044bc223\") " pod="llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp"
Sep 15 20:48:05.087062 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:05.087048    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f2deabb-1eac-4e09-a371-5b14044bc223-tls-certs\") pod \"e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp\" (UID: \"6f2deabb-1eac-4e09-a371-5b14044bc223\") " pod="llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp"
Sep 15 20:48:05.187698 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:05.187673    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f2deabb-1eac-4e09-a371-5b14044bc223-home\") pod \"e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp\" (UID: \"6f2deabb-1eac-4e09-a371-5b14044bc223\") " pod="llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp"
Sep 15 20:48:05.187816 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:05.187718    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f2deabb-1eac-4e09-a371-5b14044bc223-tls-certs\") pod \"e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp\" (UID: \"6f2deabb-1eac-4e09-a371-5b14044bc223\") " pod="llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp"
Sep 15 20:48:05.187816 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:05.187776    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f2deabb-1eac-4e09-a371-5b14044bc223-tmp-dir\") pod \"e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp\" (UID: \"6f2deabb-1eac-4e09-a371-5b14044bc223\") " pod="llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp"
Sep 15 20:48:05.187904 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:05.187815    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjlpv\" (UniqueName: \"kubernetes.io/projected/6f2deabb-1eac-4e09-a371-5b14044bc223-kube-api-access-vjlpv\") pod \"e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp\" (UID: \"6f2deabb-1eac-4e09-a371-5b14044bc223\") " pod="llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp"
Sep 15 20:48:05.187904 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:05.187839    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f2deabb-1eac-4e09-a371-5b14044bc223-dshm\") pod \"e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp\" (UID: \"6f2deabb-1eac-4e09-a371-5b14044bc223\") " pod="llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp"
Sep 15 20:48:05.188042 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:05.187983    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f2deabb-1eac-4e09-a371-5b14044bc223-model-cache\") pod \"e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp\" (UID: \"6f2deabb-1eac-4e09-a371-5b14044bc223\") " pod="llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp"
Sep 15 20:48:05.188153 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:05.188133    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f2deabb-1eac-4e09-a371-5b14044bc223-home\") pod \"e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp\" (UID: \"6f2deabb-1eac-4e09-a371-5b14044bc223\") " pod="llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp"
Sep 15 20:48:05.188318 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:05.188299    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f2deabb-1eac-4e09-a371-5b14044bc223-model-cache\") pod \"e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp\" (UID: \"6f2deabb-1eac-4e09-a371-5b14044bc223\") " pod="llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp"
Sep 15 20:48:05.188388 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:05.188373    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f2deabb-1eac-4e09-a371-5b14044bc223-tmp-dir\") pod \"e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp\" (UID: \"6f2deabb-1eac-4e09-a371-5b14044bc223\") " pod="llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp"
Sep 15 20:48:05.190109 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:05.190088    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f2deabb-1eac-4e09-a371-5b14044bc223-dshm\") pod \"e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp\" (UID: \"6f2deabb-1eac-4e09-a371-5b14044bc223\") " pod="llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp"
Sep 15 20:48:05.190391 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:05.190371    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f2deabb-1eac-4e09-a371-5b14044bc223-tls-certs\") pod \"e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp\" (UID: \"6f2deabb-1eac-4e09-a371-5b14044bc223\") " pod="llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp"
Sep 15 20:48:05.195021 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:05.195000    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjlpv\" (UniqueName: \"kubernetes.io/projected/6f2deabb-1eac-4e09-a371-5b14044bc223-kube-api-access-vjlpv\") pod \"e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp\" (UID: \"6f2deabb-1eac-4e09-a371-5b14044bc223\") " pod="llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp"
Sep 15 20:48:05.347724 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:05.347638    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp"
Sep 15 20:48:05.617143 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:05.617117    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp"]
Sep 15 20:48:05.619307 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:48:05.619279    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f2deabb_1eac_4e09_a371_5b14044bc223.slice/crio-2c0be60e49837867d9446d429745cd7f2130bac0442bed8b93ed619e6b10c877 WatchSource:0}: Error finding container 2c0be60e49837867d9446d429745cd7f2130bac0442bed8b93ed619e6b10c877: Status 404 returned error can't find the container with id 2c0be60e49837867d9446d429745cd7f2130bac0442bed8b93ed619e6b10c877
Sep 15 20:48:05.773259 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:05.773223    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp" event={"ID":"6f2deabb-1eac-4e09-a371-5b14044bc223","Type":"ContainerStarted","Data":"a1dc475f80dbf0f9c83deac76827be44ffa68071a12df11ba8e53b1b1cf274b1"}
Sep 15 20:48:05.773259 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:05.773264    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp" event={"ID":"6f2deabb-1eac-4e09-a371-5b14044bc223","Type":"ContainerStarted","Data":"2c0be60e49837867d9446d429745cd7f2130bac0442bed8b93ed619e6b10c877"}
Sep 15 20:48:05.788598 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:05.788551    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp" podStartSLOduration=0.788535886 podStartE2EDuration="788.535886ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:48:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:48:05.787484442 +0000 UTC m=+2600.275948919" watchObservedRunningTime="2026-09-15 20:48:05.788535886 +0000 UTC m=+2600.277000362"
Sep 15 20:48:15.348010 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:15.347904    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp"
Sep 15 20:48:15.348010 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:15.347975    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp"
Sep 15 20:48:15.360402 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:15.360373    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp"
Sep 15 20:48:15.834844 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:15.834815    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp"
Sep 15 20:48:19.930382 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:19.930347    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6"]
Sep 15 20:48:19.934068 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:19.934047    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6"
Sep 15 20:48:19.936420 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:19.936395    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-98d6415a-kserve-self-signed-certs\""
Sep 15 20:48:19.942210 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:19.942022    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6"]
Sep 15 20:48:19.961646 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:19.961620    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:48:20.013299 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:20.013277    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac833aba-3766-462f-be13-c6842a0b79d9-dshm\") pod \"e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6\" (UID: \"ac833aba-3766-462f-be13-c6842a0b79d9\") " pod="llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6"
Sep 15 20:48:20.013420 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:20.013328    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac833aba-3766-462f-be13-c6842a0b79d9-tls-certs\") pod \"e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6\" (UID: \"ac833aba-3766-462f-be13-c6842a0b79d9\") " pod="llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6"
Sep 15 20:48:20.013420 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:20.013381    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqb58\" (UniqueName: \"kubernetes.io/projected/ac833aba-3766-462f-be13-c6842a0b79d9-kube-api-access-kqb58\") pod \"e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6\" (UID: \"ac833aba-3766-462f-be13-c6842a0b79d9\") " pod="llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6"
Sep 15 20:48:20.013420 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:20.013398    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac833aba-3766-462f-be13-c6842a0b79d9-tmp-dir\") pod \"e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6\" (UID: \"ac833aba-3766-462f-be13-c6842a0b79d9\") " pod="llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6"
Sep 15 20:48:20.013420 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:20.013414    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac833aba-3766-462f-be13-c6842a0b79d9-home\") pod \"e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6\" (UID: \"ac833aba-3766-462f-be13-c6842a0b79d9\") " pod="llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6"
Sep 15 20:48:20.013582 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:20.013441    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac833aba-3766-462f-be13-c6842a0b79d9-model-cache\") pod \"e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6\" (UID: \"ac833aba-3766-462f-be13-c6842a0b79d9\") " pod="llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6"
Sep 15 20:48:20.114256 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:20.114225    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac833aba-3766-462f-be13-c6842a0b79d9-tls-certs\") pod \"e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6\" (UID: \"ac833aba-3766-462f-be13-c6842a0b79d9\") " pod="llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6"
Sep 15 20:48:20.114376 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:20.114270    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqb58\" (UniqueName: \"kubernetes.io/projected/ac833aba-3766-462f-be13-c6842a0b79d9-kube-api-access-kqb58\") pod \"e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6\" (UID: \"ac833aba-3766-462f-be13-c6842a0b79d9\") " pod="llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6"
Sep 15 20:48:20.114376 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:20.114290    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac833aba-3766-462f-be13-c6842a0b79d9-tmp-dir\") pod \"e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6\" (UID: \"ac833aba-3766-462f-be13-c6842a0b79d9\") " pod="llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6"
Sep 15 20:48:20.114376 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:20.114307    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac833aba-3766-462f-be13-c6842a0b79d9-home\") pod \"e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6\" (UID: \"ac833aba-3766-462f-be13-c6842a0b79d9\") " pod="llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6"
Sep 15 20:48:20.114376 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:20.114363    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac833aba-3766-462f-be13-c6842a0b79d9-model-cache\") pod \"e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6\" (UID: \"ac833aba-3766-462f-be13-c6842a0b79d9\") " pod="llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6"
Sep 15 20:48:20.114521 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:20.114446    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac833aba-3766-462f-be13-c6842a0b79d9-dshm\") pod \"e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6\" (UID: \"ac833aba-3766-462f-be13-c6842a0b79d9\") " pod="llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6"
Sep 15 20:48:20.114701 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:20.114683    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac833aba-3766-462f-be13-c6842a0b79d9-home\") pod \"e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6\" (UID: \"ac833aba-3766-462f-be13-c6842a0b79d9\") " pod="llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6"
Sep 15 20:48:20.114767 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:20.114716    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac833aba-3766-462f-be13-c6842a0b79d9-tmp-dir\") pod \"e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6\" (UID: \"ac833aba-3766-462f-be13-c6842a0b79d9\") " pod="llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6"
Sep 15 20:48:20.114820 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:20.114760    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac833aba-3766-462f-be13-c6842a0b79d9-model-cache\") pod \"e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6\" (UID: \"ac833aba-3766-462f-be13-c6842a0b79d9\") " pod="llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6"
Sep 15 20:48:20.116605 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:20.116583    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac833aba-3766-462f-be13-c6842a0b79d9-dshm\") pod \"e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6\" (UID: \"ac833aba-3766-462f-be13-c6842a0b79d9\") " pod="llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6"
Sep 15 20:48:20.116784 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:20.116765    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac833aba-3766-462f-be13-c6842a0b79d9-tls-certs\") pod \"e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6\" (UID: \"ac833aba-3766-462f-be13-c6842a0b79d9\") " pod="llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6"
Sep 15 20:48:20.121292 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:20.121267    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqb58\" (UniqueName: \"kubernetes.io/projected/ac833aba-3766-462f-be13-c6842a0b79d9-kube-api-access-kqb58\") pod \"e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6\" (UID: \"ac833aba-3766-462f-be13-c6842a0b79d9\") " pod="llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6"
Sep 15 20:48:20.245339 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:20.245309    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6"
Sep 15 20:48:20.513871 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:20.513843    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6"]
Sep 15 20:48:20.515780 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:48:20.515753    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podac833aba_3766_462f_be13_c6842a0b79d9.slice/crio-f74c30c0468ce651c5aea1f2e6c0fd729a66084665fe723e673a798143cabb59 WatchSource:0}: Error finding container f74c30c0468ce651c5aea1f2e6c0fd729a66084665fe723e673a798143cabb59: Status 404 returned error can't find the container with id f74c30c0468ce651c5aea1f2e6c0fd729a66084665fe723e673a798143cabb59
Sep 15 20:48:20.848552 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:20.848455    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6" event={"ID":"ac833aba-3766-462f-be13-c6842a0b79d9","Type":"ContainerStarted","Data":"a2e0765caf20081a4c0b41445cd397803605603c149cc7f5d661e53edf23908e"}
Sep 15 20:48:20.848552 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:20.848509    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6" event={"ID":"ac833aba-3766-462f-be13-c6842a0b79d9","Type":"ContainerStarted","Data":"f74c30c0468ce651c5aea1f2e6c0fd729a66084665fe723e673a798143cabb59"}
Sep 15 20:48:20.863567 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:20.863520    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6" podStartSLOduration=1.863504598 podStartE2EDuration="1.863504598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:48:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:48:20.862455134 +0000 UTC m=+2615.350919609" watchObservedRunningTime="2026-09-15 20:48:20.863504598 +0000 UTC m=+2615.351969074"
Sep 15 20:48:28.342672 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:28.342610    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:48:28.444125 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:28.444089    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:48:30.245885 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:30.245848    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6"
Sep 15 20:48:30.246287 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:30.245898    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6"
Sep 15 20:48:30.258191 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:30.258160    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6"
Sep 15 20:48:30.909617 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:30.909591    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6"
Sep 15 20:48:46.541861 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:46.541825    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:48:46.642671 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:46.642638    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:48:52.066324 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:52.066286    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp"]
Sep 15 20:48:52.128257 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:52.128217    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:48:53.537829 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:53.537788    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6"]
Sep 15 20:48:53.559068 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:48:53.559040    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:49:07.085807 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:07.085762    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp" podUID="6f2deabb-1eac-4e09-a371-5b14044bc223" containerName="main" containerID="cri-o://a1dc475f80dbf0f9c83deac76827be44ffa68071a12df11ba8e53b1b1cf274b1" gracePeriod=45
Sep 15 20:49:07.330994 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:07.330968    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp"
Sep 15 20:49:07.411785 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:07.411720    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjlpv\" (UniqueName: \"kubernetes.io/projected/6f2deabb-1eac-4e09-a371-5b14044bc223-kube-api-access-vjlpv\") pod \"6f2deabb-1eac-4e09-a371-5b14044bc223\" (UID: \"6f2deabb-1eac-4e09-a371-5b14044bc223\") "
Sep 15 20:49:07.411785 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:07.411760    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f2deabb-1eac-4e09-a371-5b14044bc223-tmp-dir\") pod \"6f2deabb-1eac-4e09-a371-5b14044bc223\" (UID: \"6f2deabb-1eac-4e09-a371-5b14044bc223\") "
Sep 15 20:49:07.412040 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:07.411788    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f2deabb-1eac-4e09-a371-5b14044bc223-model-cache\") pod \"6f2deabb-1eac-4e09-a371-5b14044bc223\" (UID: \"6f2deabb-1eac-4e09-a371-5b14044bc223\") "
Sep 15 20:49:07.412040 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:07.411818    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f2deabb-1eac-4e09-a371-5b14044bc223-dshm\") pod \"6f2deabb-1eac-4e09-a371-5b14044bc223\" (UID: \"6f2deabb-1eac-4e09-a371-5b14044bc223\") "
Sep 15 20:49:07.412040 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:07.411891    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f2deabb-1eac-4e09-a371-5b14044bc223-tls-certs\") pod \"6f2deabb-1eac-4e09-a371-5b14044bc223\" (UID: \"6f2deabb-1eac-4e09-a371-5b14044bc223\") "
Sep 15 20:49:07.412040 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:07.412023    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f2deabb-1eac-4e09-a371-5b14044bc223-home\") pod \"6f2deabb-1eac-4e09-a371-5b14044bc223\" (UID: \"6f2deabb-1eac-4e09-a371-5b14044bc223\") "
Sep 15 20:49:07.412232 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:07.412061    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f2deabb-1eac-4e09-a371-5b14044bc223-model-cache" (OuterVolumeSpecName: "model-cache") pod "6f2deabb-1eac-4e09-a371-5b14044bc223" (UID: "6f2deabb-1eac-4e09-a371-5b14044bc223"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:49:07.412232 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:07.412069    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f2deabb-1eac-4e09-a371-5b14044bc223-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6f2deabb-1eac-4e09-a371-5b14044bc223" (UID: "6f2deabb-1eac-4e09-a371-5b14044bc223"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:49:07.412344 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:07.412295    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f2deabb-1eac-4e09-a371-5b14044bc223-model-cache\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:49:07.412344 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:07.412320    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f2deabb-1eac-4e09-a371-5b14044bc223-tmp-dir\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:49:07.412344 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:07.412290    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f2deabb-1eac-4e09-a371-5b14044bc223-home" (OuterVolumeSpecName: "home") pod "6f2deabb-1eac-4e09-a371-5b14044bc223" (UID: "6f2deabb-1eac-4e09-a371-5b14044bc223"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:49:07.414092 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:07.414060    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f2deabb-1eac-4e09-a371-5b14044bc223-dshm" (OuterVolumeSpecName: "dshm") pod "6f2deabb-1eac-4e09-a371-5b14044bc223" (UID: "6f2deabb-1eac-4e09-a371-5b14044bc223"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:49:07.414199 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:07.414125    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f2deabb-1eac-4e09-a371-5b14044bc223-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6f2deabb-1eac-4e09-a371-5b14044bc223" (UID: "6f2deabb-1eac-4e09-a371-5b14044bc223"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:49:07.414199 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:07.414160    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f2deabb-1eac-4e09-a371-5b14044bc223-kube-api-access-vjlpv" (OuterVolumeSpecName: "kube-api-access-vjlpv") pod "6f2deabb-1eac-4e09-a371-5b14044bc223" (UID: "6f2deabb-1eac-4e09-a371-5b14044bc223"). InnerVolumeSpecName "kube-api-access-vjlpv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:49:07.513098 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:07.513066    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f2deabb-1eac-4e09-a371-5b14044bc223-tls-certs\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:49:07.513098 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:07.513091    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f2deabb-1eac-4e09-a371-5b14044bc223-home\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:49:07.513098 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:07.513101    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjlpv\" (UniqueName: \"kubernetes.io/projected/6f2deabb-1eac-4e09-a371-5b14044bc223-kube-api-access-vjlpv\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:49:07.513307 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:07.513109    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f2deabb-1eac-4e09-a371-5b14044bc223-dshm\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:49:08.080353 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:08.080320    2576 generic.go:356] "Generic (PLEG): container finished" podID="6f2deabb-1eac-4e09-a371-5b14044bc223" containerID="a1dc475f80dbf0f9c83deac76827be44ffa68071a12df11ba8e53b1b1cf274b1" exitCode=0
Sep 15 20:49:08.080545 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:08.080360    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp"
Sep 15 20:49:08.080545 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:08.080402    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp" event={"ID":"6f2deabb-1eac-4e09-a371-5b14044bc223","Type":"ContainerDied","Data":"a1dc475f80dbf0f9c83deac76827be44ffa68071a12df11ba8e53b1b1cf274b1"}
Sep 15 20:49:08.080545 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:08.080441    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp" event={"ID":"6f2deabb-1eac-4e09-a371-5b14044bc223","Type":"ContainerDied","Data":"2c0be60e49837867d9446d429745cd7f2130bac0442bed8b93ed619e6b10c877"}
Sep 15 20:49:08.080545 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:08.080459    2576 scope.go:117] "RemoveContainer" containerID="a1dc475f80dbf0f9c83deac76827be44ffa68071a12df11ba8e53b1b1cf274b1"
Sep 15 20:49:08.089837 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:08.089653    2576 scope.go:117] "RemoveContainer" containerID="a1dc475f80dbf0f9c83deac76827be44ffa68071a12df11ba8e53b1b1cf274b1"
Sep 15 20:49:08.090148 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:49:08.089896    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1dc475f80dbf0f9c83deac76827be44ffa68071a12df11ba8e53b1b1cf274b1\": container with ID starting with a1dc475f80dbf0f9c83deac76827be44ffa68071a12df11ba8e53b1b1cf274b1 not found: ID does not exist" containerID="a1dc475f80dbf0f9c83deac76827be44ffa68071a12df11ba8e53b1b1cf274b1"
Sep 15 20:49:08.090148 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:08.089917    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1dc475f80dbf0f9c83deac76827be44ffa68071a12df11ba8e53b1b1cf274b1"} err="failed to get container status \"a1dc475f80dbf0f9c83deac76827be44ffa68071a12df11ba8e53b1b1cf274b1\": rpc error: code = NotFound desc = could not find container \"a1dc475f80dbf0f9c83deac76827be44ffa68071a12df11ba8e53b1b1cf274b1\": container with ID starting with a1dc475f80dbf0f9c83deac76827be44ffa68071a12df11ba8e53b1b1cf274b1 not found: ID does not exist"
Sep 15 20:49:08.099209 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:08.099188    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp"]
Sep 15 20:49:08.105003 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:08.104982    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-98d6415a-kserve-5fddfdb7cf-mfnkp"]
Sep 15 20:49:08.142842 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:08.142816    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f2deabb-1eac-4e09-a371-5b14044bc223" path="/var/lib/kubelet/pods/6f2deabb-1eac-4e09-a371-5b14044bc223/volumes"
Sep 15 20:49:08.557240 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:08.557198    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6" podUID="ac833aba-3766-462f-be13-c6842a0b79d9" containerName="main" containerID="cri-o://a2e0765caf20081a4c0b41445cd397803605603c149cc7f5d661e53edf23908e" gracePeriod=45
Sep 15 20:49:08.803629 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:08.803601    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6"
Sep 15 20:49:08.924954 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:08.924855    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac833aba-3766-462f-be13-c6842a0b79d9-dshm\") pod \"ac833aba-3766-462f-be13-c6842a0b79d9\" (UID: \"ac833aba-3766-462f-be13-c6842a0b79d9\") "
Sep 15 20:49:08.924954 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:08.924901    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac833aba-3766-462f-be13-c6842a0b79d9-tls-certs\") pod \"ac833aba-3766-462f-be13-c6842a0b79d9\" (UID: \"ac833aba-3766-462f-be13-c6842a0b79d9\") "
Sep 15 20:49:08.924954 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:08.924945    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kqb58\" (UniqueName: \"kubernetes.io/projected/ac833aba-3766-462f-be13-c6842a0b79d9-kube-api-access-kqb58\") pod \"ac833aba-3766-462f-be13-c6842a0b79d9\" (UID: \"ac833aba-3766-462f-be13-c6842a0b79d9\") "
Sep 15 20:49:08.925223 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:08.924977    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac833aba-3766-462f-be13-c6842a0b79d9-tmp-dir\") pod \"ac833aba-3766-462f-be13-c6842a0b79d9\" (UID: \"ac833aba-3766-462f-be13-c6842a0b79d9\") "
Sep 15 20:49:08.925223 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:08.925016    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac833aba-3766-462f-be13-c6842a0b79d9-home\") pod \"ac833aba-3766-462f-be13-c6842a0b79d9\" (UID: \"ac833aba-3766-462f-be13-c6842a0b79d9\") "
Sep 15 20:49:08.925223 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:08.925159    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac833aba-3766-462f-be13-c6842a0b79d9-model-cache\") pod \"ac833aba-3766-462f-be13-c6842a0b79d9\" (UID: \"ac833aba-3766-462f-be13-c6842a0b79d9\") "
Sep 15 20:49:08.925387 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:08.925267    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac833aba-3766-462f-be13-c6842a0b79d9-home" (OuterVolumeSpecName: "home") pod "ac833aba-3766-462f-be13-c6842a0b79d9" (UID: "ac833aba-3766-462f-be13-c6842a0b79d9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:49:08.925387 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:08.925278    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac833aba-3766-462f-be13-c6842a0b79d9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ac833aba-3766-462f-be13-c6842a0b79d9" (UID: "ac833aba-3766-462f-be13-c6842a0b79d9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:49:08.925490 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:08.925387    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac833aba-3766-462f-be13-c6842a0b79d9-model-cache" (OuterVolumeSpecName: "model-cache") pod "ac833aba-3766-462f-be13-c6842a0b79d9" (UID: "ac833aba-3766-462f-be13-c6842a0b79d9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:49:08.925607 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:08.925586    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac833aba-3766-462f-be13-c6842a0b79d9-tmp-dir\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:49:08.925669 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:08.925611    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac833aba-3766-462f-be13-c6842a0b79d9-home\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:49:08.925669 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:08.925625    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac833aba-3766-462f-be13-c6842a0b79d9-model-cache\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:49:08.927059 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:08.927039    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac833aba-3766-462f-be13-c6842a0b79d9-dshm" (OuterVolumeSpecName: "dshm") pod "ac833aba-3766-462f-be13-c6842a0b79d9" (UID: "ac833aba-3766-462f-be13-c6842a0b79d9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:49:08.927172 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:08.927093    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac833aba-3766-462f-be13-c6842a0b79d9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ac833aba-3766-462f-be13-c6842a0b79d9" (UID: "ac833aba-3766-462f-be13-c6842a0b79d9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:49:08.927172 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:08.927155    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac833aba-3766-462f-be13-c6842a0b79d9-kube-api-access-kqb58" (OuterVolumeSpecName: "kube-api-access-kqb58") pod "ac833aba-3766-462f-be13-c6842a0b79d9" (UID: "ac833aba-3766-462f-be13-c6842a0b79d9"). InnerVolumeSpecName "kube-api-access-kqb58". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:49:09.026110 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:09.026083    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac833aba-3766-462f-be13-c6842a0b79d9-dshm\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:49:09.026243 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:09.026139    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac833aba-3766-462f-be13-c6842a0b79d9-tls-certs\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:49:09.026243 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:09.026166    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqb58\" (UniqueName: \"kubernetes.io/projected/ac833aba-3766-462f-be13-c6842a0b79d9-kube-api-access-kqb58\") on node \"ip-10-0-134-155.ec2.internal\" DevicePath \"\""
Sep 15 20:49:09.087651 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:09.087617    2576 generic.go:356] "Generic (PLEG): container finished" podID="ac833aba-3766-462f-be13-c6842a0b79d9" containerID="a2e0765caf20081a4c0b41445cd397803605603c149cc7f5d661e53edf23908e" exitCode=0
Sep 15 20:49:09.087782 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:09.087666    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6"
Sep 15 20:49:09.087782 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:09.087682    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6" event={"ID":"ac833aba-3766-462f-be13-c6842a0b79d9","Type":"ContainerDied","Data":"a2e0765caf20081a4c0b41445cd397803605603c149cc7f5d661e53edf23908e"}
Sep 15 20:49:09.087782 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:09.087718    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6" event={"ID":"ac833aba-3766-462f-be13-c6842a0b79d9","Type":"ContainerDied","Data":"f74c30c0468ce651c5aea1f2e6c0fd729a66084665fe723e673a798143cabb59"}
Sep 15 20:49:09.087782 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:09.087739    2576 scope.go:117] "RemoveContainer" containerID="a2e0765caf20081a4c0b41445cd397803605603c149cc7f5d661e53edf23908e"
Sep 15 20:49:09.096951 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:09.096772    2576 scope.go:117] "RemoveContainer" containerID="a2e0765caf20081a4c0b41445cd397803605603c149cc7f5d661e53edf23908e"
Sep 15 20:49:09.097168 ip-10-0-134-155 kubenswrapper[2576]: E0915 20:49:09.097080    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2e0765caf20081a4c0b41445cd397803605603c149cc7f5d661e53edf23908e\": container with ID starting with a2e0765caf20081a4c0b41445cd397803605603c149cc7f5d661e53edf23908e not found: ID does not exist" containerID="a2e0765caf20081a4c0b41445cd397803605603c149cc7f5d661e53edf23908e"
Sep 15 20:49:09.097168 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:09.097108    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2e0765caf20081a4c0b41445cd397803605603c149cc7f5d661e53edf23908e"} err="failed to get container status \"a2e0765caf20081a4c0b41445cd397803605603c149cc7f5d661e53edf23908e\": rpc error: code = NotFound desc = could not find container \"a2e0765caf20081a4c0b41445cd397803605603c149cc7f5d661e53edf23908e\": container with ID starting with a2e0765caf20081a4c0b41445cd397803605603c149cc7f5d661e53edf23908e not found: ID does not exist"
Sep 15 20:49:09.109138 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:09.109115    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6"]
Sep 15 20:49:09.112520 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:09.112502    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-98d6415a-kserve-67bf8cc48c-8xqx6"]
Sep 15 20:49:10.142416 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:10.142385    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac833aba-3766-462f-be13-c6842a0b79d9" path="/var/lib/kubelet/pods/ac833aba-3766-462f-be13-c6842a0b79d9/volumes"
Sep 15 20:49:24.644432 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:24.644395    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6b4c4886c6-7nbpm"]
Sep 15 20:49:24.644971 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:24.644772    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac833aba-3766-462f-be13-c6842a0b79d9" containerName="main"
Sep 15 20:49:24.644971 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:24.644785    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac833aba-3766-462f-be13-c6842a0b79d9" containerName="main"
Sep 15 20:49:24.644971 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:24.644806    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f2deabb-1eac-4e09-a371-5b14044bc223" containerName="main"
Sep 15 20:49:24.644971 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:24.644811    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f2deabb-1eac-4e09-a371-5b14044bc223" containerName="main"
Sep 15 20:49:24.644971 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:24.644875    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac833aba-3766-462f-be13-c6842a0b79d9" containerName="main"
Sep 15 20:49:24.644971 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:24.644886    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f2deabb-1eac-4e09-a371-5b14044bc223" containerName="main"
Sep 15 20:49:24.647784 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:24.647769    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-7nbpm"
Sep 15 20:49:24.650292 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:24.650264    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 20:49:24.650860 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:24.650844    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 15 20:49:24.650955 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:24.650881    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-76rfp\""
Sep 15 20:49:24.651127 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:24.651111    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 15 20:49:24.651222 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:24.651204    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 15 20:49:24.657666 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:24.657645    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b4c4886c6-7nbpm"]
Sep 15 20:49:24.765015 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:24.764989    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58f3ec27-01a0-4a4a-8e93-a1e34b2f574d-metrics-certs\") pod \"maas-api-6b4c4886c6-7nbpm\" (UID: \"58f3ec27-01a0-4a4a-8e93-a1e34b2f574d\") " pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-7nbpm"
Sep 15 20:49:24.765157 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:24.765029    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b829p\" (UniqueName: \"kubernetes.io/projected/58f3ec27-01a0-4a4a-8e93-a1e34b2f574d-kube-api-access-b829p\") pod \"maas-api-6b4c4886c6-7nbpm\" (UID: \"58f3ec27-01a0-4a4a-8e93-a1e34b2f574d\") " pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-7nbpm"
Sep 15 20:49:24.765157 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:24.765057    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58f3ec27-01a0-4a4a-8e93-a1e34b2f574d-maas-api-tls\") pod \"maas-api-6b4c4886c6-7nbpm\" (UID: \"58f3ec27-01a0-4a4a-8e93-a1e34b2f574d\") " pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-7nbpm"
Sep 15 20:49:24.765286 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:24.765169    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58f3ec27-01a0-4a4a-8e93-a1e34b2f574d-openshift-service-ca\") pod \"maas-api-6b4c4886c6-7nbpm\" (UID: \"58f3ec27-01a0-4a4a-8e93-a1e34b2f574d\") " pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-7nbpm"
Sep 15 20:49:24.865561 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:24.865537    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58f3ec27-01a0-4a4a-8e93-a1e34b2f574d-openshift-service-ca\") pod \"maas-api-6b4c4886c6-7nbpm\" (UID: \"58f3ec27-01a0-4a4a-8e93-a1e34b2f574d\") " pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-7nbpm"
Sep 15 20:49:24.865685 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:24.865583    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58f3ec27-01a0-4a4a-8e93-a1e34b2f574d-metrics-certs\") pod \"maas-api-6b4c4886c6-7nbpm\" (UID: \"58f3ec27-01a0-4a4a-8e93-a1e34b2f574d\") " pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-7nbpm"
Sep 15 20:49:24.865754 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:24.865705    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b829p\" (UniqueName: \"kubernetes.io/projected/58f3ec27-01a0-4a4a-8e93-a1e34b2f574d-kube-api-access-b829p\") pod \"maas-api-6b4c4886c6-7nbpm\" (UID: \"58f3ec27-01a0-4a4a-8e93-a1e34b2f574d\") " pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-7nbpm"
Sep 15 20:49:24.865812 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:24.865772    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58f3ec27-01a0-4a4a-8e93-a1e34b2f574d-maas-api-tls\") pod \"maas-api-6b4c4886c6-7nbpm\" (UID: \"58f3ec27-01a0-4a4a-8e93-a1e34b2f574d\") " pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-7nbpm"
Sep 15 20:49:24.866227 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:24.866206    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58f3ec27-01a0-4a4a-8e93-a1e34b2f574d-openshift-service-ca\") pod \"maas-api-6b4c4886c6-7nbpm\" (UID: \"58f3ec27-01a0-4a4a-8e93-a1e34b2f574d\") " pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-7nbpm"
Sep 15 20:49:24.868049 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:24.868030    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58f3ec27-01a0-4a4a-8e93-a1e34b2f574d-maas-api-tls\") pod \"maas-api-6b4c4886c6-7nbpm\" (UID: \"58f3ec27-01a0-4a4a-8e93-a1e34b2f574d\") " pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-7nbpm"
Sep 15 20:49:24.868049 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:24.868039    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58f3ec27-01a0-4a4a-8e93-a1e34b2f574d-metrics-certs\") pod \"maas-api-6b4c4886c6-7nbpm\" (UID: \"58f3ec27-01a0-4a4a-8e93-a1e34b2f574d\") " pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-7nbpm"
Sep 15 20:49:24.872857 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:24.872839    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b829p\" (UniqueName: \"kubernetes.io/projected/58f3ec27-01a0-4a4a-8e93-a1e34b2f574d-kube-api-access-b829p\") pod \"maas-api-6b4c4886c6-7nbpm\" (UID: \"58f3ec27-01a0-4a4a-8e93-a1e34b2f574d\") " pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-7nbpm"
Sep 15 20:49:24.958388 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:24.958367    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-7nbpm"
Sep 15 20:49:25.227744 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:25.227716    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b4c4886c6-7nbpm"]
Sep 15 20:49:25.229152 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:49:25.229130    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58f3ec27_01a0_4a4a_8e93_a1e34b2f574d.slice/crio-dd7a97105fd1abf474fe6ae2b5b08b5f2ee67efe0b50e2101e7fceb2db6af4c9 WatchSource:0}: Error finding container dd7a97105fd1abf474fe6ae2b5b08b5f2ee67efe0b50e2101e7fceb2db6af4c9: Status 404 returned error can't find the container with id dd7a97105fd1abf474fe6ae2b5b08b5f2ee67efe0b50e2101e7fceb2db6af4c9
Sep 15 20:49:25.230915 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:25.230894    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 20:49:26.172293 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:26.172248    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-7nbpm" event={"ID":"58f3ec27-01a0-4a4a-8e93-a1e34b2f574d","Type":"ContainerStarted","Data":"dd7a97105fd1abf474fe6ae2b5b08b5f2ee67efe0b50e2101e7fceb2db6af4c9"}
Sep 15 20:49:28.182406 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:28.182363    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-7nbpm" event={"ID":"58f3ec27-01a0-4a4a-8e93-a1e34b2f574d","Type":"ContainerStarted","Data":"1ea0e34be1bf57fadc1fa2ef36dc2194330f3c93db028e0a05d769c170ea28d0"}
Sep 15 20:49:28.182812 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:28.182491    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-7nbpm"
Sep 15 20:49:28.205970 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:28.205428    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-7nbpm" podStartSLOduration=1.945506982 podStartE2EDuration="4.205408715s" totalImagesPullingTime="2.259901733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:49:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:49:28.203315817 +0000 UTC m=+2682.691780305" watchObservedRunningTime="2026-09-15 20:49:28.205408715 +0000 UTC m=+2682.693873192"
Sep 15 20:49:34.193131 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:34.193097    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-7nbpm"
Sep 15 20:49:46.525490 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:46.525418    2576 scope.go:117] "RemoveContainer" containerID="6b74f36afc5e633342c830f912a3933611b260ddcdcecb3615844087b61e6141"
Sep 15 20:49:46.533462 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:46.533441    2576 scope.go:117] "RemoveContainer" containerID="1983e12c4e27976beb04116939a29ef99f986d98d7ebdb6848ad3ff67304d58b"
Sep 15 20:49:46.540588 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:46.540570    2576 scope.go:117] "RemoveContainer" containerID="ddb5c658a1f961e993ea31e3f4986a3fedba25d067eb118d35e1e80bc61a95d8"
Sep 15 20:49:46.617341 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:46.617315    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-b469d9fb-c98sk_38e3235c-f4c7-4404-9470-a9cc0f6f763a/authorino/0.log"
Sep 15 20:49:52.736194 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:52.736166    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-75fcf566-t6w5p_8f45205a-57e4-4819-931e-f3533ee73a92/copy-manifests/0.log"
Sep 15 20:49:52.742264 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:52.742227    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-75fcf566-t6w5p_8f45205a-57e4-4819-931e-f3533ee73a92/manager/0.log"
Sep 15 20:49:53.067663 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:53.067571    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6b4c4886c6-7nbpm_58f3ec27-01a0-4a4a-8e93-a1e34b2f574d/maas-api/0.log"
Sep 15 20:49:53.380058 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:53.379983    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29825085-tgcdz_72939fb6-0f65-43cc-b956-e212029fbd27/cleanup/0.log"
Sep 15 20:49:54.252513 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:54.252479    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-b469d9fb-c98sk_38e3235c-f4c7-4404-9470-a9cc0f6f763a/authorino/0.log"
Sep 15 20:49:54.959310 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:54.959277    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-dfzbk_0ed6bbae-de4f-41ce-ae10-0cd3175cb10e/manager/0.log"
Sep 15 20:49:55.534271 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:55.534243    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-565c49d7c-bgkjg_a98edf6b-4657-4ae2-b251-596b6ac44cf2/kube-auth-proxy/0.log"
Sep 15 20:49:55.872874 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:55.872801    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-869x2_0599d238-3847-4c55-89c3-c3074e1d9ad0/payload-pre-processing/0.log"
Sep 15 20:49:56.095403 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:56.095376    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-b6b4698b-54vk2_085814ed-8800-43b7-bc59-b4f46cf14608/router/0.log"
Sep 15 20:49:56.433766 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:56.433737    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-pk27s_52855211-f580-4ab1-87b7-edbf73326b5c/main/0.log"
Sep 15 20:49:56.762418 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:56.762388    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-8vtfv_e0bfa23e-9b3f-42cf-b040-1c97b27fbe69/main/0.log"
Sep 15 20:49:56.993773 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:56.993743    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-2bd8x_0629cd62-1aae-406d-9ce1-d8e6e964a624/main/0.log"
Sep 15 20:49:57.121120 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:49:57.121024    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-vlb4m_81d98002-64a0-4881-b58a-5eba5848ff97/main/0.log"
Sep 15 20:50:02.841031 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:02.840996    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:50:02.970637 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:02.970585    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:50:05.631682 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:05.631649    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7hmwc_6c3054ad-31af-4d06-9809-1eab5e2b140c/global-pull-secret-syncer/0.log"
Sep 15 20:50:05.753356 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:05.753322    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log"
Sep 15 20:50:05.842233 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:05.842203    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-155.ec2.internal_6cec6a6594652885b2e8d4cf7af01cc1/haproxy/0.log"
Sep 15 20:50:10.297607 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:10.297579    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-b469d9fb-c98sk_38e3235c-f4c7-4404-9470-a9cc0f6f763a/authorino/0.log"
Sep 15 20:50:10.523439 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:10.523407    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-dfzbk_0ed6bbae-de4f-41ce-ae10-0cd3175cb10e/manager/0.log"
Sep 15 20:50:12.349468 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:12.349442    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-wdnhk_c085a1d9-7bd7-4769-9b58-c554276913f8/cluster-monitoring-operator/0.log"
Sep 15 20:50:12.380542 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:12.380516    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-shd92_ffe68d8d-4681-4754-bf5c-a0ef28de8037/kube-state-metrics/0.log"
Sep 15 20:50:12.398329 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:12.398291    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-shd92_ffe68d8d-4681-4754-bf5c-a0ef28de8037/kube-rbac-proxy-main/0.log"
Sep 15 20:50:12.418025 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:12.418005    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-shd92_ffe68d8d-4681-4754-bf5c-a0ef28de8037/kube-rbac-proxy-self/0.log"
Sep 15 20:50:12.638334 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:12.638309    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tvkc6_69d39979-106e-4316-91e4-6dfa7546a724/node-exporter/0.log"
Sep 15 20:50:12.660433 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:12.660408    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tvkc6_69d39979-106e-4316-91e4-6dfa7546a724/kube-rbac-proxy/0.log"
Sep 15 20:50:12.679445 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:12.679427    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tvkc6_69d39979-106e-4316-91e4-6dfa7546a724/init-textfile/0.log"
Sep 15 20:50:12.703562 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:12.703544    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-cb4km_2f6a0afa-7e21-48f1-9173-6a0fbc184be7/kube-rbac-proxy-main/0.log"
Sep 15 20:50:12.722760 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:12.722744    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-cb4km_2f6a0afa-7e21-48f1-9173-6a0fbc184be7/kube-rbac-proxy-self/0.log"
Sep 15 20:50:12.741457 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:12.741442    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-cb4km_2f6a0afa-7e21-48f1-9173-6a0fbc184be7/openshift-state-metrics/0.log"
Sep 15 20:50:12.775666 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:12.775645    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7850e2c8-449b-43b9-bdc3-896a75717f66/prometheus/0.log"
Sep 15 20:50:12.796021 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:12.795998    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7850e2c8-449b-43b9-bdc3-896a75717f66/config-reloader/0.log"
Sep 15 20:50:12.815806 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:12.815789    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7850e2c8-449b-43b9-bdc3-896a75717f66/thanos-sidecar/0.log"
Sep 15 20:50:12.834570 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:12.834554    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7850e2c8-449b-43b9-bdc3-896a75717f66/kube-rbac-proxy-web/0.log"
Sep 15 20:50:12.858075 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:12.858056    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7850e2c8-449b-43b9-bdc3-896a75717f66/kube-rbac-proxy/0.log"
Sep 15 20:50:12.877648 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:12.877596    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7850e2c8-449b-43b9-bdc3-896a75717f66/kube-rbac-proxy-thanos/0.log"
Sep 15 20:50:12.896840 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:12.896818    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7850e2c8-449b-43b9-bdc3-896a75717f66/init-config-reloader/0.log"
Sep 15 20:50:12.925934 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:12.925907    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-s9xjv_a58414ce-141f-46ee-b7de-a3735aae50f7/prometheus-operator/0.log"
Sep 15 20:50:12.945536 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:12.945514    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-s9xjv_a58414ce-141f-46ee-b7de-a3735aae50f7/kube-rbac-proxy/0.log"
Sep 15 20:50:12.967544 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:12.967524    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-87qsf_33eaf6a7-ab22-48a4-9502-3fc7943c9c27/prometheus-operator-admission-webhook/0.log"
Sep 15 20:50:14.841825 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:14.841796    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-2kszh_fcb6cd63-75f0-4efd-a878-fb7236f4109b/networking-console-plugin/0.log"
Sep 15 20:50:16.215809 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:16.215778    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8bqd6/perf-node-gather-daemonset-s9gcb"]
Sep 15 20:50:16.222763 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:16.222744    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-s9gcb"
Sep 15 20:50:16.225502 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:16.225476    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8bqd6\"/\"openshift-service-ca.crt\""
Sep 15 20:50:16.225643 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:16.225539    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8bqd6\"/\"default-dockercfg-fgwsj\""
Sep 15 20:50:16.226285 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:16.226262    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8bqd6\"/\"kube-root-ca.crt\""
Sep 15 20:50:16.226551 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:16.226530    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8bqd6/perf-node-gather-daemonset-s9gcb"]
Sep 15 20:50:16.315425 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:16.315393    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a0927118-e610-4c3a-a151-1ea3352bb98e-podres\") pod \"perf-node-gather-daemonset-s9gcb\" (UID: \"a0927118-e610-4c3a-a151-1ea3352bb98e\") " pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-s9gcb"
Sep 15 20:50:16.315594 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:16.315433    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fx8d\" (UniqueName: \"kubernetes.io/projected/a0927118-e610-4c3a-a151-1ea3352bb98e-kube-api-access-8fx8d\") pod \"perf-node-gather-daemonset-s9gcb\" (UID: \"a0927118-e610-4c3a-a151-1ea3352bb98e\") " pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-s9gcb"
Sep 15 20:50:16.315594 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:16.315466    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a0927118-e610-4c3a-a151-1ea3352bb98e-lib-modules\") pod \"perf-node-gather-daemonset-s9gcb\" (UID: \"a0927118-e610-4c3a-a151-1ea3352bb98e\") " pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-s9gcb"
Sep 15 20:50:16.315594 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:16.315543    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a0927118-e610-4c3a-a151-1ea3352bb98e-proc\") pod \"perf-node-gather-daemonset-s9gcb\" (UID: \"a0927118-e610-4c3a-a151-1ea3352bb98e\") " pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-s9gcb"
Sep 15 20:50:16.315719 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:16.315601    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a0927118-e610-4c3a-a151-1ea3352bb98e-sys\") pod \"perf-node-gather-daemonset-s9gcb\" (UID: \"a0927118-e610-4c3a-a151-1ea3352bb98e\") " pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-s9gcb"
Sep 15 20:50:16.416507 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:16.416477    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a0927118-e610-4c3a-a151-1ea3352bb98e-proc\") pod \"perf-node-gather-daemonset-s9gcb\" (UID: \"a0927118-e610-4c3a-a151-1ea3352bb98e\") " pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-s9gcb"
Sep 15 20:50:16.416507 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:16.416509    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a0927118-e610-4c3a-a151-1ea3352bb98e-sys\") pod \"perf-node-gather-daemonset-s9gcb\" (UID: \"a0927118-e610-4c3a-a151-1ea3352bb98e\") " pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-s9gcb"
Sep 15 20:50:16.416678 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:16.416565    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a0927118-e610-4c3a-a151-1ea3352bb98e-podres\") pod \"perf-node-gather-daemonset-s9gcb\" (UID: \"a0927118-e610-4c3a-a151-1ea3352bb98e\") " pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-s9gcb"
Sep 15 20:50:16.416678 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:16.416579    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fx8d\" (UniqueName: \"kubernetes.io/projected/a0927118-e610-4c3a-a151-1ea3352bb98e-kube-api-access-8fx8d\") pod \"perf-node-gather-daemonset-s9gcb\" (UID: \"a0927118-e610-4c3a-a151-1ea3352bb98e\") " pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-s9gcb"
Sep 15 20:50:16.416678 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:16.416587    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a0927118-e610-4c3a-a151-1ea3352bb98e-proc\") pod \"perf-node-gather-daemonset-s9gcb\" (UID: \"a0927118-e610-4c3a-a151-1ea3352bb98e\") " pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-s9gcb"
Sep 15 20:50:16.416678 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:16.416600    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a0927118-e610-4c3a-a151-1ea3352bb98e-lib-modules\") pod \"perf-node-gather-daemonset-s9gcb\" (UID: \"a0927118-e610-4c3a-a151-1ea3352bb98e\") " pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-s9gcb"
Sep 15 20:50:16.416678 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:16.416636    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a0927118-e610-4c3a-a151-1ea3352bb98e-sys\") pod \"perf-node-gather-daemonset-s9gcb\" (UID: \"a0927118-e610-4c3a-a151-1ea3352bb98e\") " pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-s9gcb"
Sep 15 20:50:16.416838 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:16.416711    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a0927118-e610-4c3a-a151-1ea3352bb98e-lib-modules\") pod \"perf-node-gather-daemonset-s9gcb\" (UID: \"a0927118-e610-4c3a-a151-1ea3352bb98e\") " pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-s9gcb"
Sep 15 20:50:16.416838 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:16.416718    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a0927118-e610-4c3a-a151-1ea3352bb98e-podres\") pod \"perf-node-gather-daemonset-s9gcb\" (UID: \"a0927118-e610-4c3a-a151-1ea3352bb98e\") " pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-s9gcb"
Sep 15 20:50:16.423826 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:16.423802    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fx8d\" (UniqueName: \"kubernetes.io/projected/a0927118-e610-4c3a-a151-1ea3352bb98e-kube-api-access-8fx8d\") pod \"perf-node-gather-daemonset-s9gcb\" (UID: \"a0927118-e610-4c3a-a151-1ea3352bb98e\") " pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-s9gcb"
Sep 15 20:50:16.533357 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:16.533298    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-s9gcb"
Sep 15 20:50:16.800403 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:16.800377    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8bqd6/perf-node-gather-daemonset-s9gcb"]
Sep 15 20:50:16.801909 ip-10-0-134-155 kubenswrapper[2576]: W0915 20:50:16.801879    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda0927118_e610_4c3a_a151_1ea3352bb98e.slice/crio-bdcc6f42f65540e1e3e4f52253a30b563babbdc57b24e8af946e3e831bba83c2 WatchSource:0}: Error finding container bdcc6f42f65540e1e3e4f52253a30b563babbdc57b24e8af946e3e831bba83c2: Status 404 returned error can't find the container with id bdcc6f42f65540e1e3e4f52253a30b563babbdc57b24e8af946e3e831bba83c2
Sep 15 20:50:17.422281 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:17.422241    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-s9gcb" event={"ID":"a0927118-e610-4c3a-a151-1ea3352bb98e","Type":"ContainerStarted","Data":"ad9896aa46c13f18136164b5c5dcd1948a2e1af27b43f4683efd0e2c2f7646b9"}
Sep 15 20:50:17.422665 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:17.422300    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-s9gcb"
Sep 15 20:50:17.422665 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:17.422316    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-s9gcb" event={"ID":"a0927118-e610-4c3a-a151-1ea3352bb98e","Type":"ContainerStarted","Data":"bdcc6f42f65540e1e3e4f52253a30b563babbdc57b24e8af946e3e831bba83c2"}
Sep 15 20:50:17.440198 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:17.440151    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-s9gcb" podStartSLOduration=1.440135334 podStartE2EDuration="1.440135334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:50:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:50:17.438779639 +0000 UTC m=+2731.927244117" watchObservedRunningTime="2026-09-15 20:50:17.440135334 +0000 UTC m=+2731.928599865"
Sep 15 20:50:18.079219 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:18.079192    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qg2bs_0c65a532-9608-4066-9b1a-156727664921/dns/0.log"
Sep 15 20:50:18.097652 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:18.097630    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qg2bs_0c65a532-9608-4066-9b1a-156727664921/kube-rbac-proxy/0.log"
Sep 15 20:50:18.146212 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:18.146185    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-s8ddr_6cd01117-433d-4273-a167-01344ad4f018/dns-node-resolver/0.log"
Sep 15 20:50:18.830765 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:18.830730    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qk9ld_48d7dc57-6de6-4a72-8da3-156c90d7a8ff/node-ca/0.log"
Sep 15 20:50:20.267505 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:20.267468    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-565c49d7c-bgkjg_a98edf6b-4657-4ae2-b251-596b6ac44cf2/kube-auth-proxy/0.log"
Sep 15 20:50:20.358756 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:20.358724    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-869x2_0599d238-3847-4c55-89c3-c3074e1d9ad0/payload-pre-processing/0.log"
Sep 15 20:50:20.421069 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:20.421044    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-b6b4698b-54vk2_085814ed-8800-43b7-bc59-b4f46cf14608/router/0.log"
Sep 15 20:50:21.107936 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:21.107889    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vdw2_e6bec389-ca68-4845-8694-9571cf84fe3f/konnectivity-agent/0.log" limit=1024
Sep 15 20:50:21.349186 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:21.349158    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-49mwc_7a1d40a8-4253-43b6-b7c8-d278b7649dcd/serve-healthcheck-canary/0.log"
Sep 15 20:50:21.856594 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:21.856566    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-9cxxk_93c1a30d-050c-491d-adbc-b5e876968efd/insights-operator/0.log"
Sep 15 20:50:21.857066 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:21.857050    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-9cxxk_93c1a30d-050c-491d-adbc-b5e876968efd/insights-operator/1.log"
Sep 15 20:50:21.876458 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:21.876427    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ns6zw_d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f/kube-rbac-proxy/0.log"
Sep 15 20:50:21.896241 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:21.896221    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ns6zw_d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f/exporter/0.log"
Sep 15 20:50:21.915723 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:21.915706    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ns6zw_d2cf9195-fe5f-41ef-8d06-65de3e0e4e3f/extractor/0.log"
Sep 15 20:50:23.437302 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:23.437272    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-s9gcb"
Sep 15 20:50:24.816131 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:24.816105    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-75fcf566-t6w5p_8f45205a-57e4-4819-931e-f3533ee73a92/manager/0.log"
Sep 15 20:50:24.836139 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:24.836117    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-75fcf566-t6w5p_8f45205a-57e4-4819-931e-f3533ee73a92/copy-manifests/0.log"
Sep 15 20:50:26.325283 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:26.325249    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-d95cdffcd-lg6sh_a13421ec-d464-4435-8b63-76298a3becb0/manager/0.log"
Sep 15 20:50:27.810220 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:27.810189    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-565c49d7c-bgkjg_a98edf6b-4657-4ae2-b251-596b6ac44cf2/kube-auth-proxy/0.log"
Sep 15 20:50:27.866480 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:27.866402    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-869x2_0599d238-3847-4c55-89c3-c3074e1d9ad0/payload-pre-processing/0.log"
Sep 15 20:50:27.906704 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:27.906677    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-b6b4698b-54vk2_085814ed-8800-43b7-bc59-b4f46cf14608/router/0.log"
Sep 15 20:50:29.935008 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:29.934972    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-565c49d7c-bgkjg_a98edf6b-4657-4ae2-b251-596b6ac44cf2/kube-auth-proxy/0.log"
Sep 15 20:50:30.002403 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:30.002375    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-869x2_0599d238-3847-4c55-89c3-c3074e1d9ad0/payload-pre-processing/0.log"
Sep 15 20:50:30.049672 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:30.049645    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-b6b4698b-54vk2_085814ed-8800-43b7-bc59-b4f46cf14608/router/0.log"
Sep 15 20:50:33.019083 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:33.019055    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9r4qs_794a8849-e693-4fd1-905d-addff4740cda/kube-multus-additional-cni-plugins/0.log"
Sep 15 20:50:33.039288 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:33.039256    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9r4qs_794a8849-e693-4fd1-905d-addff4740cda/egress-router-binary-copy/0.log"
Sep 15 20:50:33.058193 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:33.058176    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9r4qs_794a8849-e693-4fd1-905d-addff4740cda/cni-plugins/0.log"
Sep 15 20:50:33.078131 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:33.078114    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9r4qs_794a8849-e693-4fd1-905d-addff4740cda/bond-cni-plugin/0.log"
Sep 15 20:50:33.098061 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:33.098039    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9r4qs_794a8849-e693-4fd1-905d-addff4740cda/routeoverride-cni/0.log"
Sep 15 20:50:33.117144 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:33.117101    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9r4qs_794a8849-e693-4fd1-905d-addff4740cda/whereabouts-cni-bincopy/0.log"
Sep 15 20:50:33.135469 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:33.135441    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9r4qs_794a8849-e693-4fd1-905d-addff4740cda/whereabouts-cni/0.log"
Sep 15 20:50:33.585888 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:33.585849    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-f7npj_5368fcd5-c6e8-4f29-b93c-338fdd857bad/kube-multus/0.log"
Sep 15 20:50:33.650929 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:33.650893    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-59mfs_3b8d9342-6574-4d0d-a326-0299eb3b4c5d/network-metrics-daemon/0.log"
Sep 15 20:50:33.668418 ip-10-0-134-155 kubenswrapper[2576]: I0915 20:50:33.668380    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-59mfs_3b8d9342-6574-4d0d-a326-0299eb3b4c5d/kube-rbac-proxy/0.log"