Sep 12 00:27:24.515016 ip-10-0-142-85 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 12 00:27:24.515030 ip-10-0-142-85 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 12 00:27:24.515039 ip-10-0-142-85 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 12 00:27:24.515367 ip-10-0-142-85 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 12 00:27:34.635175 ip-10-0-142-85 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 12 00:27:34.635194 ip-10-0-142-85 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot d1f89b877aee48ba92d9d4b19a7c85f4 --
Sep 12 00:29:23.227014 ip-10-0-142-85 systemd[1]: Starting Kubernetes Kubelet...
Sep 12 00:29:23.552347 ip-10-0-142-85 kubenswrapper[2558]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 12 00:29:23.552347 ip-10-0-142-85 kubenswrapper[2558]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 12 00:29:23.552347 ip-10-0-142-85 kubenswrapper[2558]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 12 00:29:23.552347 ip-10-0-142-85 kubenswrapper[2558]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 12 00:29:23.552347 ip-10-0-142-85 kubenswrapper[2558]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 12 00:29:23.553176 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.553070    2558 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 12 00:29:23.557397 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557379    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 12 00:29:23.557397 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557396    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 12 00:29:23.557462 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557400    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 12 00:29:23.557462 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557403    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 12 00:29:23.557462 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557406    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 12 00:29:23.557462 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557409    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 12 00:29:23.557462 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557413    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 12 00:29:23.557462 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557416    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 12 00:29:23.557462 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557419    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 12 00:29:23.557462 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557422    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 12 00:29:23.557462 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557426    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 12 00:29:23.557462 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557429    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 12 00:29:23.557462 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557432    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 12 00:29:23.557462 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557435    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 12 00:29:23.557462 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557437    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 12 00:29:23.557462 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557440    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 12 00:29:23.557462 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557444    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 12 00:29:23.557462 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557446    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 12 00:29:23.557462 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557449    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 12 00:29:23.557462 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557451    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 12 00:29:23.557462 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557454    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 12 00:29:23.557462 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557456    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 12 00:29:23.557936 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557459    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 12 00:29:23.557936 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557461    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 12 00:29:23.557936 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557464    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 12 00:29:23.557936 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557467    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 12 00:29:23.557936 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557469    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 12 00:29:23.557936 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557472    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 12 00:29:23.557936 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557475    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 12 00:29:23.557936 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557478    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 12 00:29:23.557936 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557480    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 12 00:29:23.557936 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557483    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 12 00:29:23.557936 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557486    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 12 00:29:23.557936 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557488    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 12 00:29:23.557936 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557490    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 12 00:29:23.557936 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557493    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 12 00:29:23.557936 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557495    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 12 00:29:23.557936 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557498    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 12 00:29:23.557936 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557500    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 12 00:29:23.557936 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557502    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 12 00:29:23.557936 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557505    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 12 00:29:23.557936 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557508    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 12 00:29:23.558437 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557510    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 12 00:29:23.558437 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557513    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 12 00:29:23.558437 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557515    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 12 00:29:23.558437 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557517    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 12 00:29:23.558437 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557520    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 12 00:29:23.558437 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557522    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 12 00:29:23.558437 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557525    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 12 00:29:23.558437 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557527    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 12 00:29:23.558437 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557530    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 12 00:29:23.558437 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557532    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 12 00:29:23.558437 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557535    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 12 00:29:23.558437 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557537    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 12 00:29:23.558437 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557539    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 12 00:29:23.558437 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557543    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 12 00:29:23.558437 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557547    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 12 00:29:23.558437 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557551    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 12 00:29:23.558437 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557554    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 12 00:29:23.558437 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557556    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 12 00:29:23.558437 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557561    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 12 00:29:23.558935 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557566    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 12 00:29:23.558935 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557569    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 12 00:29:23.558935 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557573    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 12 00:29:23.558935 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557577    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 12 00:29:23.558935 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557580    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 12 00:29:23.558935 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557583    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 12 00:29:23.558935 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557586    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 12 00:29:23.558935 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557588    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 12 00:29:23.558935 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557591    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 12 00:29:23.558935 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557594    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 12 00:29:23.558935 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557597    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 12 00:29:23.558935 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557600    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 12 00:29:23.558935 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557602    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 12 00:29:23.558935 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557605    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 12 00:29:23.558935 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557608    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 12 00:29:23.558935 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557610    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 12 00:29:23.558935 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557613    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 12 00:29:23.558935 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557616    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 12 00:29:23.558935 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557618    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 12 00:29:23.559416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557621    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 12 00:29:23.559416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557626    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 12 00:29:23.559416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557629    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 12 00:29:23.559416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557631    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 12 00:29:23.559416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557634    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 12 00:29:23.559416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557637    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 12 00:29:23.559416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557639    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 12 00:29:23.559416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557642    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 12 00:29:23.559416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557645    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 12 00:29:23.559416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557647    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 12 00:29:23.559416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557650    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 12 00:29:23.559416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557652    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 12 00:29:23.559416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557655    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 12 00:29:23.559416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557658    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 12 00:29:23.559416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557660    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 12 00:29:23.559416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557663    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 12 00:29:23.559416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557666    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 12 00:29:23.559416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557668    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 12 00:29:23.559416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.557671    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 12 00:29:23.559416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558397    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 12 00:29:23.559913 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558404    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 12 00:29:23.559913 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558407    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 12 00:29:23.559913 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558410    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 12 00:29:23.559913 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558412    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 12 00:29:23.559913 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558417    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 12 00:29:23.559913 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558421    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 12 00:29:23.559913 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558424    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 12 00:29:23.559913 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558428    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 12 00:29:23.559913 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558430    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 12 00:29:23.559913 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558433    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 12 00:29:23.559913 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558436    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 12 00:29:23.559913 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558439    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 12 00:29:23.559913 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558443    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 12 00:29:23.559913 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558445    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 12 00:29:23.559913 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558448    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 12 00:29:23.559913 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558450    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 12 00:29:23.559913 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558453    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 12 00:29:23.559913 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558457    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 12 00:29:23.559913 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558459    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 12 00:29:23.559913 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558464    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 12 00:29:23.560443 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558468    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 12 00:29:23.560443 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558471    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 12 00:29:23.560443 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558474    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 12 00:29:23.560443 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558477    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 12 00:29:23.560443 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558479    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 12 00:29:23.560443 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558482    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 12 00:29:23.560443 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558485    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 12 00:29:23.560443 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558488    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 12 00:29:23.560443 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558491    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 12 00:29:23.560443 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558494    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 12 00:29:23.560443 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558497    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 12 00:29:23.560443 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558499    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 12 00:29:23.560443 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558501    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 12 00:29:23.560443 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558504    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 12 00:29:23.560443 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558506    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 12 00:29:23.560443 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558509    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 12 00:29:23.560443 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558511    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 12 00:29:23.560443 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558514    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 12 00:29:23.560443 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558518    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 12 00:29:23.560443 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558521    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 12 00:29:23.560924 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558524    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 12 00:29:23.560924 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558526    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 12 00:29:23.560924 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558528    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 12 00:29:23.560924 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558531    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 12 00:29:23.560924 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558533    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 12 00:29:23.560924 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558535    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 12 00:29:23.560924 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558538    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 12 00:29:23.560924 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558540    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 12 00:29:23.560924 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558543    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 12 00:29:23.560924 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558545    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 12 00:29:23.560924 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558548    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 12 00:29:23.560924 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558552    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 12 00:29:23.560924 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558554    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 12 00:29:23.560924 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558557    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 12 00:29:23.560924 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558559    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 12 00:29:23.560924 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558562    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 12 00:29:23.560924 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558565    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 12 00:29:23.560924 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558567    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 12 00:29:23.560924 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558571    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 12 00:29:23.560924 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558573    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 12 00:29:23.561419 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558576    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 12 00:29:23.561419 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558579    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 12 00:29:23.561419 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558581    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 12 00:29:23.561419 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558584    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 12 00:29:23.561419 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558586    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 12 00:29:23.561419 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558589    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 12 00:29:23.561419 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558591    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 12 00:29:23.561419 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558594    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 12 00:29:23.561419 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558596    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 12 00:29:23.561419 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558598    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 12 00:29:23.561419 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558601    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 12 00:29:23.561419 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558603    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 12 00:29:23.561419 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558606    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 12 00:29:23.561419 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558608    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 12 00:29:23.561419 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558611    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 12 00:29:23.561419 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558613    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 12 00:29:23.561419 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558615    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 12 00:29:23.561419 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558618    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 12 00:29:23.561419 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558620    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 12 00:29:23.561882 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558623    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 12 00:29:23.561882 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558625    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 12 00:29:23.561882 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558627    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 12 00:29:23.561882 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558630    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 12 00:29:23.561882 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558633    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 12 00:29:23.561882 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558636    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 12 00:29:23.561882 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558639    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 12 00:29:23.561882 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558641    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 12 00:29:23.561882 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558644    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 12 00:29:23.561882 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558646    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 12 00:29:23.561882 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558649    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 12 00:29:23.561882 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558651    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 12 00:29:23.561882 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558654    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 12 00:29:23.561882 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558656    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 12 00:29:23.561882 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558658    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 12 00:29:23.561882 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558661    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 12 00:29:23.561882 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558664    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 12 00:29:23.561882 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558667    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 12 00:29:23.561882 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.558669    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 12 00:29:23.562359 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559100    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 12 00:29:23.562359 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559106    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 12 00:29:23.562359 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559109    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 12 00:29:23.562359 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559112    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 12 00:29:23.562359 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559115    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 12 00:29:23.562359 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559117    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 12 00:29:23.562359 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559120    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 12 00:29:23.562359 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559123    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 12 00:29:23.562359 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559125    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 12 00:29:23.562359 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559128    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 12 00:29:23.562359 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559130    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 12 00:29:23.562359 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559132    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 12 00:29:23.562359 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559135    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 12 00:29:23.562359 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559138    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 12 00:29:23.562359 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559141    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 12 00:29:23.562359 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559144    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 12 00:29:23.562359 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559146    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 12 00:29:23.562359 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559149    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 12 00:29:23.562359 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559152    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 12 00:29:23.562821 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559154    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 12 00:29:23.562821 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559157    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 12 00:29:23.562821 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559159    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 12 00:29:23.562821 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559161    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 12 00:29:23.562821 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559164    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 12 00:29:23.562821 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559168    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 12 00:29:23.562821 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559172    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 12 00:29:23.562821 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559175    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 12 00:29:23.562821 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559177    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 12 00:29:23.562821 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559180    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 12 00:29:23.562821 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559183    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 12 00:29:23.562821 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559185    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 12 00:29:23.562821 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559188    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 12 00:29:23.562821 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559191    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 12 00:29:23.562821 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559193    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 12 00:29:23.562821 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559196    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 12 00:29:23.562821 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559199    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 12 00:29:23.562821 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559202    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 12 00:29:23.562821 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559204    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 12 00:29:23.563310 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559207    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 12 00:29:23.563310 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559209    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 12 00:29:23.563310 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559212    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 12 00:29:23.563310 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559214    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 12 00:29:23.563310 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559217    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 12 00:29:23.563310 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559219    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 12 00:29:23.563310 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559221    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 12 00:29:23.563310 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559225    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 12 00:29:23.563310 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559228    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 12 00:29:23.563310 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559231    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 12 00:29:23.563310 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559233    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 12 00:29:23.563310 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559235    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 12 00:29:23.563310 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559238    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 12 00:29:23.563310 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559240    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 12 00:29:23.563310 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559243    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 12 00:29:23.563310 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559245    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 12 00:29:23.563310 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559248    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 12 00:29:23.563310 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559250    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 12 00:29:23.563310 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559253    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 12 00:29:23.563310 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559255    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 12 00:29:23.563848 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559258    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 12 00:29:23.563848 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559260    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 12 00:29:23.563848 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559262    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 12 00:29:23.563848 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559268    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 12 00:29:23.563848 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559271    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 12 00:29:23.563848 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559273    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 12 00:29:23.563848 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559276    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 12 00:29:23.563848 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559278    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 12 00:29:23.563848 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559281    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 12 00:29:23.563848 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559283    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 12 00:29:23.563848 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559285    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 12 00:29:23.563848 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559288    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 12 00:29:23.563848 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559290    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 12 00:29:23.563848 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559292    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 12 00:29:23.563848 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559295    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 12 00:29:23.563848 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559297    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 12 00:29:23.563848 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559299    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 12 00:29:23.563848 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559302    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 12 00:29:23.563848 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559306    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 12 00:29:23.563848 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559311    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 12 00:29:23.564365 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559314    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 12 00:29:23.564365 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559317    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 12 00:29:23.564365 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559320    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 12 00:29:23.564365 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559322    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 12 00:29:23.564365 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559326    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 12 00:29:23.564365 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559331    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 12 00:29:23.564365 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559334    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 12 00:29:23.564365 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559337    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 12 00:29:23.564365 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559340    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 12 00:29:23.564365 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559344    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 12 00:29:23.564365 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559347    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 12 00:29:23.564365 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559350    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 12 00:29:23.564365 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559352    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 12 00:29:23.564365 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559355    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 12 00:29:23.564365 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559357    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 12 00:29:23.564365 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559360    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 12 00:29:23.564365 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559363    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 12 00:29:23.564365 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559365    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 12 00:29:23.564365 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559368    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 12 00:29:23.564365 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559370    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 12 00:29:23.564850 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.559372    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 12 00:29:23.564850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559445    2558 flags.go:64] FLAG: --address="0.0.0.0"
Sep 12 00:29:23.564850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559479    2558 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 12 00:29:23.564850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559486    2558 flags.go:64] FLAG: --anonymous-auth="true"
Sep 12 00:29:23.564850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559498    2558 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 12 00:29:23.564850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559503    2558 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 12 00:29:23.564850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559506    2558 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 12 00:29:23.564850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559511    2558 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 12 00:29:23.564850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559515    2558 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 12 00:29:23.564850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559518    2558 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 12 00:29:23.564850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559521    2558 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 12 00:29:23.564850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559528    2558 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 12 00:29:23.564850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559531    2558 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 12 00:29:23.564850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559534    2558 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 12 00:29:23.564850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559538    2558 flags.go:64] FLAG: --cgroup-root=""
Sep 12 00:29:23.564850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559541    2558 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 12 00:29:23.564850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559544    2558 flags.go:64] FLAG: --client-ca-file=""
Sep 12 00:29:23.564850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559548    2558 flags.go:64] FLAG: --cloud-provider="external"
Sep 12 00:29:23.564850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559551    2558 flags.go:64] FLAG: --cluster-dns="[]"
Sep 12 00:29:23.564850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559555    2558 flags.go:64] FLAG: --cluster-domain=""
Sep 12 00:29:23.564850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559558    2558 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 12 00:29:23.564850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559561    2558 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 12 00:29:23.564850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559564    2558 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 12 00:29:23.564850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559568    2558 flags.go:64] FLAG: --container-log-max-files="5"
Sep 12 00:29:23.565437 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559572    2558 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 12 00:29:23.565437 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559575    2558 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 12 00:29:23.565437 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559578    2558 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 12 00:29:23.565437 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559582    2558 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 12 00:29:23.565437 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559585    2558 flags.go:64] FLAG: --contention-profiling="false"
Sep 12 00:29:23.565437 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559589    2558 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 12 00:29:23.565437 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559592    2558 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 12 00:29:23.565437 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559595    2558 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 12 00:29:23.565437 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559598    2558 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 12 00:29:23.565437 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559602    2558 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 12 00:29:23.565437 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559605    2558 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 12 00:29:23.565437 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559608    2558 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 12 00:29:23.565437 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559611    2558 flags.go:64] FLAG: --enable-load-reader="false"
Sep 12 00:29:23.565437 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559614    2558 flags.go:64] FLAG: --enable-server="true"
Sep 12 00:29:23.565437 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559617    2558 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 12 00:29:23.565437 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559622    2558 flags.go:64] FLAG: --event-burst="100"
Sep 12 00:29:23.565437 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559625    2558 flags.go:64] FLAG: --event-qps="50"
Sep 12 00:29:23.565437 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559628    2558 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 12 00:29:23.565437 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559631    2558 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 12 00:29:23.565437 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559635    2558 flags.go:64] FLAG: --eviction-hard=""
Sep 12 00:29:23.565437 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559638    2558 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 12 00:29:23.565437 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559642    2558 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 12 00:29:23.565437 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559645    2558 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 12 00:29:23.565437 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559647    2558 flags.go:64] FLAG: --eviction-soft=""
Sep 12 00:29:23.565437 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559650    2558 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 12 00:29:23.566135 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559655    2558 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 12 00:29:23.566135 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559658    2558 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 12 00:29:23.566135 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559661    2558 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 12 00:29:23.566135 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559664    2558 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 12 00:29:23.566135 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559667    2558 flags.go:64] FLAG: --fail-swap-on="true"
Sep 12 00:29:23.566135 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559670    2558 flags.go:64] FLAG: --feature-gates=""
Sep 12 00:29:23.566135 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559673    2558 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 12 00:29:23.566135 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559676    2558 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 12 00:29:23.566135 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559680    2558 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 12 00:29:23.566135 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559683    2558 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 12 00:29:23.566135 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559686    2558 flags.go:64] FLAG: --healthz-port="10248"
Sep 12 00:29:23.566135 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559689    2558 flags.go:64] FLAG: --help="false"
Sep 12 00:29:23.566135 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559693    2558 flags.go:64] FLAG: --hostname-override="ip-10-0-142-85.ec2.internal"
Sep 12 00:29:23.566135 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559696    2558 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 12 00:29:23.566135 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559699    2558 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 12 00:29:23.566135 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559702    2558 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 12 00:29:23.566135 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559705    2558 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 12 00:29:23.566135 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559708    2558 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 12 00:29:23.566135 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559711    2558 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 12 00:29:23.566135 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559714    2558 flags.go:64] FLAG: --image-service-endpoint=""
Sep 12 00:29:23.566135 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559716    2558 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 12 00:29:23.566135 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559719    2558 flags.go:64] FLAG: --kube-api-burst="100"
Sep 12 00:29:23.566135 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559722    2558 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 12 00:29:23.566721 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559725    2558 flags.go:64] FLAG: --kube-api-qps="50"
Sep 12 00:29:23.566721 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559728    2558 flags.go:64] FLAG: --kube-reserved=""
Sep 12 00:29:23.566721 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559731    2558 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 12 00:29:23.566721 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559735    2558 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 12 00:29:23.566721 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559738    2558 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 12 00:29:23.566721 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559741    2558 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 12 00:29:23.566721 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559744    2558 flags.go:64] FLAG: --lock-file=""
Sep 12 00:29:23.566721 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559747    2558 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 12 00:29:23.566721 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559750    2558 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 12 00:29:23.566721 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559754    2558 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 12 00:29:23.566721 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559759    2558 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 12 00:29:23.566721 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559762    2558 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 12 00:29:23.566721 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559765    2558 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 12 00:29:23.566721 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559767    2558 flags.go:64] FLAG: --logging-format="text"
Sep 12 00:29:23.566721 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559771    2558 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 12 00:29:23.566721 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559774    2558 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 12 00:29:23.566721 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559777    2558 flags.go:64] FLAG: --manifest-url=""
Sep 12 00:29:23.566721 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559780    2558 flags.go:64] FLAG: --manifest-url-header=""
Sep 12 00:29:23.566721 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559784    2558 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 12 00:29:23.566721 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559787    2558 flags.go:64] FLAG: --max-open-files="1000000"
Sep 12 00:29:23.566721 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559791    2558 flags.go:64] FLAG: --max-pods="110"
Sep 12 00:29:23.566721 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559794    2558 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 12 00:29:23.566721 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559797    2558 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 12 00:29:23.566721 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559801    2558 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 12 00:29:23.566721 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559804    2558 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 12 00:29:23.567375 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559807    2558 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 12 00:29:23.567375 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559810    2558 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 12 00:29:23.567375 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559813    2558 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 12 00:29:23.567375 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559820    2558 flags.go:64] FLAG: --node-status-max-images="50"
Sep 12 00:29:23.567375 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559823    2558 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 12 00:29:23.567375 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559826    2558 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 12 00:29:23.567375 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559829    2558 flags.go:64] FLAG: --pod-cidr=""
Sep 12 00:29:23.567375 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559832    2558 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 12 00:29:23.567375 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559838    2558 flags.go:64] FLAG: --pod-manifest-path=""
Sep 12 00:29:23.567375 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559840    2558 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 12 00:29:23.567375 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559844    2558 flags.go:64] FLAG: --pods-per-core="0"
Sep 12 00:29:23.567375 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559848    2558 flags.go:64] FLAG: --port="10250"
Sep 12 00:29:23.567375 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559851    2558 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 12 00:29:23.567375 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559853    2558 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-017b18ba31cc5e7ea"
Sep 12 00:29:23.567375 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559857    2558 flags.go:64] FLAG: --qos-reserved=""
Sep 12 00:29:23.567375 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559860    2558 flags.go:64] FLAG: --read-only-port="10255"
Sep 12 00:29:23.567375 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559864    2558 flags.go:64] FLAG: --register-node="true"
Sep 12 00:29:23.567375 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559867    2558 flags.go:64] FLAG: --register-with-taints=""
Sep 12 00:29:23.567375 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559871    2558 flags.go:64] FLAG: --registry-burst="10"
Sep 12 00:29:23.567375 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559874    2558 flags.go:64] FLAG: --registry-qps="5"
Sep 12 00:29:23.567375 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559877    2558 flags.go:64] FLAG: --reserved-cpus=""
Sep 12 00:29:23.567375 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559879    2558 flags.go:64] FLAG: --reserved-memory=""
Sep 12 00:29:23.567375 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559883    2558 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 12 00:29:23.567375 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559886    2558 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 12 00:29:23.567375 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559889    2558 flags.go:64] FLAG: --rotate-certificates="false"
Sep 12 00:29:23.567996 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559892    2558 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 12 00:29:23.567996 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559895    2558 flags.go:64] FLAG: --runonce="false"
Sep 12 00:29:23.567996 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559898    2558 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 12 00:29:23.567996 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559901    2558 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 12 00:29:23.567996 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559904    2558 flags.go:64] FLAG: --seccomp-default="false"
Sep 12 00:29:23.567996 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559907    2558 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 12 00:29:23.567996 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559911    2558 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 12 00:29:23.567996 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559914    2558 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 12 00:29:23.567996 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559917    2558 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 12 00:29:23.567996 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559921    2558 flags.go:64] FLAG: --storage-driver-password="root"
Sep 12 00:29:23.567996 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559924    2558 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 12 00:29:23.567996 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559927    2558 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 12 00:29:23.567996 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559930    2558 flags.go:64] FLAG: --storage-driver-user="root"
Sep 12 00:29:23.567996 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559933    2558 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 12 00:29:23.567996 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559936    2558 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 12 00:29:23.567996 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559939    2558 flags.go:64] FLAG: --system-cgroups=""
Sep 12 00:29:23.567996 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559942    2558 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 12 00:29:23.567996 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559947    2558 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 12 00:29:23.567996 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559951    2558 flags.go:64] FLAG: --tls-cert-file=""
Sep 12 00:29:23.567996 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559954    2558 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 12 00:29:23.567996 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559957    2558 flags.go:64] FLAG: --tls-min-version=""
Sep 12 00:29:23.567996 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559960    2558 flags.go:64] FLAG: --tls-private-key-file=""
Sep 12 00:29:23.567996 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559963    2558 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 12 00:29:23.567996 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559966    2558 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 12 00:29:23.567996 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559969    2558 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 12 00:29:23.568639 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559972    2558 flags.go:64] FLAG: --v="2"
Sep 12 00:29:23.568639 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559976    2558 flags.go:64] FLAG: --version="false"
Sep 12 00:29:23.568639 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559980    2558 flags.go:64] FLAG: --vmodule=""
Sep 12 00:29:23.568639 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559983    2558 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 12 00:29:23.568639 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.559987    2558 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 12 00:29:23.568639 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560118    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 12 00:29:23.568639 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560124    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 12 00:29:23.568639 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560127    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 12 00:29:23.568639 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560131    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 12 00:29:23.568639 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560134    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 12 00:29:23.568639 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560137    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 12 00:29:23.568639 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560139    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 12 00:29:23.568639 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560142    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 12 00:29:23.568639 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560145    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 12 00:29:23.568639 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560148    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 12 00:29:23.568639 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560151    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 12 00:29:23.568639 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560153    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 12 00:29:23.568639 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560156    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 12 00:29:23.568639 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560158    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 12 00:29:23.568639 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560161    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 12 00:29:23.568639 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560163    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 12 00:29:23.569416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560166    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 12 00:29:23.569416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560168    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 12 00:29:23.569416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560171    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 12 00:29:23.569416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560173    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 12 00:29:23.569416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560176    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 12 00:29:23.569416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560179    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 12 00:29:23.569416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560181    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 12 00:29:23.569416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560184    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 12 00:29:23.569416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560187    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 12 00:29:23.569416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560190    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 12 00:29:23.569416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560192    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 12 00:29:23.569416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560195    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 12 00:29:23.569416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560197    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 12 00:29:23.569416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560200    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 12 00:29:23.569416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560202    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 12 00:29:23.569416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560205    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 12 00:29:23.569416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560207    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 12 00:29:23.569416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560210    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 12 00:29:23.569416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560212    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 12 00:29:23.569416 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560214    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 12 00:29:23.569943 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560217    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 12 00:29:23.569943 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560219    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 12 00:29:23.569943 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560222    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 12 00:29:23.569943 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560224    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 12 00:29:23.569943 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560227    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 12 00:29:23.569943 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560230    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 12 00:29:23.569943 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560233    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 12 00:29:23.569943 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560235    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 12 00:29:23.569943 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560238    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 12 00:29:23.569943 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560240    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 12 00:29:23.569943 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560242    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 12 00:29:23.569943 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560245    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 12 00:29:23.569943 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560247    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 12 00:29:23.569943 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560257    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 12 00:29:23.569943 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560262    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 12 00:29:23.569943 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560265    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 12 00:29:23.569943 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560270    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 12 00:29:23.569943 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560272    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 12 00:29:23.569943 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560275    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 12 00:29:23.570433 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560277    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 12 00:29:23.570433 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560280    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 12 00:29:23.570433 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560283    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 12 00:29:23.570433 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560285    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 12 00:29:23.570433 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560288    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 12 00:29:23.570433 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560291    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 12 00:29:23.570433 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560293    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 12 00:29:23.570433 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560296    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 12 00:29:23.570433 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560298    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 12 00:29:23.570433 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560301    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 12 00:29:23.570433 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560303    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 12 00:29:23.570433 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560306    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 12 00:29:23.570433 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560308    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 12 00:29:23.570433 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560311    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 12 00:29:23.570433 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560313    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 12 00:29:23.570433 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560316    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 12 00:29:23.570433 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560318    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 12 00:29:23.570433 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560320    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 12 00:29:23.570433 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560323    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 12 00:29:23.570433 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560325    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 12 00:29:23.570918 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560328    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 12 00:29:23.570918 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560331    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 12 00:29:23.570918 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560334    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 12 00:29:23.570918 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560336    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 12 00:29:23.570918 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560340    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 12 00:29:23.570918 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560343    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 12 00:29:23.570918 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560346    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 12 00:29:23.570918 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560349    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 12 00:29:23.570918 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560352    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 12 00:29:23.570918 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560354    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 12 00:29:23.570918 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560357    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 12 00:29:23.570918 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560359    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 12 00:29:23.570918 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560362    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 12 00:29:23.570918 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560364    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 12 00:29:23.570918 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560367    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 12 00:29:23.570918 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560369    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 12 00:29:23.570918 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560372    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 12 00:29:23.570918 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560374    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 12 00:29:23.570918 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560377    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 12 00:29:23.571439 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560379    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 12 00:29:23.571439 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560381    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 12 00:29:23.571439 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560384    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 12 00:29:23.571439 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560387    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 12 00:29:23.571439 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.560391    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 12 00:29:23.571439 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.560988    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 12 00:29:23.571439 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.571133    2558 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 12 00:29:23.571439 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.571150    2558 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 12 00:29:23.571880 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.571869    2558 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 12 00:29:23.571912 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.571880    2558 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 12 00:29:23.571949 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.571941    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 12 00:29:23.571949 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.571949    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 12 00:29:23.572002 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.571953    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 12 00:29:23.572002 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.571956    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 12 00:29:23.572002 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.571959    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 12 00:29:23.572002 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.571962    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 12 00:29:23.572002 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.571966    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 12 00:29:23.572002 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.571970    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 12 00:29:23.572002 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.571974    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 12 00:29:23.572002 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.571976    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 12 00:29:23.572002 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.571979    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 12 00:29:23.572002 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.571982    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 12 00:29:23.572002 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.571986    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 12 00:29:23.572002 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.571990    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 12 00:29:23.572002 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.571993    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 12 00:29:23.572002 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.571996    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 12 00:29:23.572002 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.571999    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 12 00:29:23.572002 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572002    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 12 00:29:23.572002 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572005    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 12 00:29:23.572002 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572008    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 12 00:29:23.572002 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572011    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 12 00:29:23.572558 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572014    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 12 00:29:23.572558 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572017    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 12 00:29:23.572558 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572019    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 12 00:29:23.572558 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572022    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 12 00:29:23.572558 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572025    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 12 00:29:23.572558 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572028    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 12 00:29:23.572558 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572031    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 12 00:29:23.572558 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572033    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 12 00:29:23.572558 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572036    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 12 00:29:23.572558 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572038    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 12 00:29:23.572558 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572041    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 12 00:29:23.572558 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572043    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 12 00:29:23.572558 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572046    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 12 00:29:23.572558 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572049    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 12 00:29:23.572558 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572051    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 12 00:29:23.572558 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572054    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 12 00:29:23.572558 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572056    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 12 00:29:23.572558 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572058    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 12 00:29:23.572558 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572061    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 12 00:29:23.572558 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572063    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 12 00:29:23.573029 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572066    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 12 00:29:23.573029 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572068    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 12 00:29:23.573029 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572071    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 12 00:29:23.573029 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572074    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 12 00:29:23.573029 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572076    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 12 00:29:23.573029 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572079    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 12 00:29:23.573029 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572098    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 12 00:29:23.573029 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572101    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 12 00:29:23.573029 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572103    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 12 00:29:23.573029 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572106    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 12 00:29:23.573029 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572108    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 12 00:29:23.573029 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572111    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 12 00:29:23.573029 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572114    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 12 00:29:23.573029 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572117    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 12 00:29:23.573029 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572120    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 12 00:29:23.573029 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572122    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 12 00:29:23.573029 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572125    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 12 00:29:23.573029 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572128    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 12 00:29:23.573029 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572130    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 12 00:29:23.573029 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572133    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 12 00:29:23.573522 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572136    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 12 00:29:23.573522 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572138    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 12 00:29:23.573522 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572142    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 12 00:29:23.573522 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572144    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 12 00:29:23.573522 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572147    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 12 00:29:23.573522 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572149    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 12 00:29:23.573522 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572152    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 12 00:29:23.573522 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572154    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 12 00:29:23.573522 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572157    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 12 00:29:23.573522 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572159    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 12 00:29:23.573522 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572162    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 12 00:29:23.573522 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572164    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 12 00:29:23.573522 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572167    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 12 00:29:23.573522 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572169    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 12 00:29:23.573522 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572172    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 12 00:29:23.573522 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572174    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 12 00:29:23.573522 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572177    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 12 00:29:23.573522 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572179    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 12 00:29:23.573522 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572188    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 12 00:29:23.573985 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572192    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 12 00:29:23.573985 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572195    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 12 00:29:23.573985 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572198    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 12 00:29:23.573985 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572200    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 12 00:29:23.573985 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572203    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 12 00:29:23.573985 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572207    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 12 00:29:23.573985 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572210    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 12 00:29:23.573985 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572213    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 12 00:29:23.573985 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572216    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 12 00:29:23.573985 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572219    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 12 00:29:23.573985 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572222    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 12 00:29:23.573985 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572224    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 12 00:29:23.573985 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572227    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 12 00:29:23.573985 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572230    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 12 00:29:23.573985 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572232    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 12 00:29:23.573985 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572235    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 12 00:29:23.573985 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572238    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 12 00:29:23.573985 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572240    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 12 00:29:23.573985 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572243    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 12 00:29:23.574459 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.572248    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 12 00:29:23.574459 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572373    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 12 00:29:23.574459 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572379    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 12 00:29:23.574459 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572382    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 12 00:29:23.574459 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572385    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 12 00:29:23.574459 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572387    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 12 00:29:23.574459 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572390    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 12 00:29:23.574459 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572393    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 12 00:29:23.574459 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572396    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 12 00:29:23.574459 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572399    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 12 00:29:23.574459 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572402    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 12 00:29:23.574459 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572404    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 12 00:29:23.574459 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572407    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 12 00:29:23.574459 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572410    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 12 00:29:23.574459 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572412    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 12 00:29:23.574834 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572414    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 12 00:29:23.574834 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572417    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 12 00:29:23.574834 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572419    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 12 00:29:23.574834 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572422    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 12 00:29:23.574834 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572425    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 12 00:29:23.574834 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572428    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 12 00:29:23.574834 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572430    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 12 00:29:23.574834 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572433    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 12 00:29:23.574834 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572436    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 12 00:29:23.574834 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572438    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 12 00:29:23.574834 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572441    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 12 00:29:23.574834 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572443    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 12 00:29:23.574834 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572446    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 12 00:29:23.574834 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572449    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 12 00:29:23.574834 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572451    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 12 00:29:23.574834 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572453    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 12 00:29:23.574834 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572456    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 12 00:29:23.574834 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572458    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 12 00:29:23.574834 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572460    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 12 00:29:23.575323 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572464    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 12 00:29:23.575323 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572467    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 12 00:29:23.575323 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572469    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 12 00:29:23.575323 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572472    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 12 00:29:23.575323 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572474    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 12 00:29:23.575323 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572477    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 12 00:29:23.575323 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572481    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 12 00:29:23.575323 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572485    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 12 00:29:23.575323 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572488    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 12 00:29:23.575323 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572491    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 12 00:29:23.575323 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572495    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 12 00:29:23.575323 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572498    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 12 00:29:23.575323 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572502    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 12 00:29:23.575323 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572505    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 12 00:29:23.575323 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572508    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 12 00:29:23.575323 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572510    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 12 00:29:23.575323 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572513    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 12 00:29:23.575323 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572515    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 12 00:29:23.575323 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572518    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 12 00:29:23.575829 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572521    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 12 00:29:23.575829 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572523    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 12 00:29:23.575829 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572526    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 12 00:29:23.575829 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572528    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 12 00:29:23.575829 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572531    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 12 00:29:23.575829 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572534    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 12 00:29:23.575829 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572536    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 12 00:29:23.575829 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572539    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 12 00:29:23.575829 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572541    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 12 00:29:23.575829 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572545    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 12 00:29:23.575829 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572548    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 12 00:29:23.575829 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572550    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 12 00:29:23.575829 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572553    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 12 00:29:23.575829 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572555    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 12 00:29:23.575829 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572558    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 12 00:29:23.575829 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572561    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 12 00:29:23.575829 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572563    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 12 00:29:23.575829 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572566    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 12 00:29:23.575829 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572568    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 12 00:29:23.576333 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572571    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 12 00:29:23.576333 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572574    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 12 00:29:23.576333 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572577    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 12 00:29:23.576333 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572579    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 12 00:29:23.576333 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572582    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 12 00:29:23.576333 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572584    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 12 00:29:23.576333 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572587    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 12 00:29:23.576333 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572590    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 12 00:29:23.576333 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572592    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 12 00:29:23.576333 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572595    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 12 00:29:23.576333 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572597    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 12 00:29:23.576333 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572600    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 12 00:29:23.576333 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572603    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 12 00:29:23.576333 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572605    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 12 00:29:23.576333 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572608    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 12 00:29:23.576333 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572610    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 12 00:29:23.576333 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572613    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 12 00:29:23.576333 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572615    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 12 00:29:23.576333 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572618    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 12 00:29:23.576333 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572620    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 12 00:29:23.577139 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572623    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 12 00:29:23.577139 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572625    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 12 00:29:23.577139 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572627    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 12 00:29:23.577139 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572630    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 12 00:29:23.577139 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572632    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 12 00:29:23.577139 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572635    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 12 00:29:23.577139 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572637    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 12 00:29:23.577139 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:23.572640    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 12 00:29:23.577139 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.572645    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 12 00:29:23.577139 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.573299    2558 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 12 00:29:23.577139 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.575397    2558 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 12 00:29:23.577139 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.576285    2558 server.go:1019] "Starting client certificate rotation"
Sep 12 00:29:23.577139 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.576397    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 12 00:29:23.577139 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.576450    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 12 00:29:23.579100 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.579069    2558 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 12 00:29:23.580289 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.580268    2558 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 12 00:29:23.590777 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.590756    2558 log.go:25] "Validated CRI v1 runtime API"
Sep 12 00:29:23.593839 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.593822    2558 log.go:25] "Validated CRI v1 image API"
Sep 12 00:29:23.594609 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.594596    2558 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 12 00:29:23.597491 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.597458    2558 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 ca12b095-5b39-4510-a94a-1899e21856ce:/dev/nvme0n1p3 f3c70610-3a75-477a-98b9-4f9e5034827a:/dev/nvme0n1p4]
Sep 12 00:29:23.597584 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.597491    2558 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 12 00:29:23.602641 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.602530    2558 manager.go:216] Machine: {Timestamp:2026-09-12 00:29:23.601249977 +0000 UTC m=+0.275308827 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102638 MemoryCapacity:32812158976 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27f449e63654052277164081024400 SystemUUID:ec27f449-e636-5405-2277-164081024400 BootID:d1f89b87-7aee-48ba-92d9-d4b19a7c85f4 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b6:d9:09:e1:bf Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b6:d9:09:e1:bf Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c6:5d:5a:c7:33:33 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812158976 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 12 00:29:23.602641 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.602630    2558 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 12 00:29:23.602772 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.602710    2558 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 12 00:29:23.605211 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.605188    2558 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 12 00:29:23.605342 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.605215    2558 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-85.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 12 00:29:23.605947 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.605938    2558 topology_manager.go:138] "Creating topology manager with none policy"
Sep 12 00:29:23.605978 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.605949    2558 container_manager_linux.go:309] "Creating device plugin manager"
Sep 12 00:29:23.605978 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.605957    2558 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 12 00:29:23.607288 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.607276    2558 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 12 00:29:23.608073 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.608064    2558 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 12 00:29:23.610001 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.609979    2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 12 00:29:23.611378 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.611365    2558 state_mem.go:36] "Initialized new in-memory state store"
Sep 12 00:29:23.611485 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.611476    2558 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 12 00:29:23.611518 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.611506    2558 kubelet.go:486] "Attempting to sync node with API server"
Sep 12 00:29:23.611518 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.611516    2558 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 12 00:29:23.611575 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.611531    2558 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 12 00:29:23.611575 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.611544    2558 kubelet.go:398] "Adding apiserver pod source"
Sep 12 00:29:23.611575 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.611556    2558 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 12 00:29:23.611951 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.611939    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 12 00:29:23.612001 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.611965    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 12 00:29:23.613846 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.613816    2558 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 12 00:29:23.616051 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.616037    2558 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 12 00:29:23.616113 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.616069    2558 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 12 00:29:23.618028 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.618013    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 12 00:29:23.618099 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.618035    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 12 00:29:23.618099 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.618041    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 12 00:29:23.618099 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.618047    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 12 00:29:23.618099 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.618055    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 12 00:29:23.618099 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.618060    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 12 00:29:23.618099 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.618073    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 12 00:29:23.618099 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.618079    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 12 00:29:23.618294 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.618107    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 12 00:29:23.618294 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.618113    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 12 00:29:23.618294 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.618121    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 12 00:29:23.618294 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.618127    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 12 00:29:23.618920 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.618906    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 12 00:29:23.618952 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.618922    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 12 00:29:23.622737 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.622723    2558 server.go:1268] "Started kubelet"
Sep 12 00:29:23.622818 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.622793    2558 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 12 00:29:23.623180 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.622974    2558 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 12 00:29:23.623296 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:23.623238    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 12 00:29:23.623414 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:23.623393    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-85.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 12 00:29:23.623482 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.623449    2558 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 12 00:29:23.623482 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.623468    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-85.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 12 00:29:23.623796 ip-10-0-142-85 systemd[1]: Started Kubernetes Kubelet.
Sep 12 00:29:23.625864 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.625843    2558 server.go:310] "Adding debug handlers to kubelet server"
Sep 12 00:29:23.628721 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.628702    2558 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 12 00:29:23.629827 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:23.629790    2558 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 12 00:29:23.630577 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:23.629612    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-85.ec2.internal.18d46b54ba762e1c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-85.ec2.internal,UID:ip-10-0-142-85.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-85.ec2.internal,},FirstTimestamp:2026-09-12 00:29:23.62266166 +0000 UTC m=+0.296720510,LastTimestamp:2026-09-12 00:29:23.62266166 +0000 UTC m=+0.296720510,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-85.ec2.internal,}"
Sep 12 00:29:23.632781 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.632766    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6swql"
Sep 12 00:29:23.633130 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.633114    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 12 00:29:23.633684 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.633656    2558 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 12 00:29:23.633989 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:23.633967    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-85.ec2.internal\" not found"
Sep 12 00:29:23.634246 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.634230    2558 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 12 00:29:23.634246 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.634234    2558 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 12 00:29:23.634346 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.634255    2558 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 12 00:29:23.634392 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.634382    2558 reconstruct.go:97] "Volume reconstruction finished"
Sep 12 00:29:23.634392 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.634389    2558 reconciler.go:29] "Reconciler: start to sync state"
Sep 12 00:29:23.634815 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.634801    2558 factory.go:55] Registering systemd factory
Sep 12 00:29:23.634879 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.634818    2558 factory.go:223] Registration of the systemd container factory successfully
Sep 12 00:29:23.635034 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.635023    2558 factory.go:153] Registering CRI-O factory
Sep 12 00:29:23.635034 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.635034    2558 factory.go:223] Registration of the crio container factory successfully
Sep 12 00:29:23.635161 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.635072    2558 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 12 00:29:23.635161 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.635108    2558 factory.go:103] Registering Raw factory
Sep 12 00:29:23.635161 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.635121    2558 manager.go:1196] Started watching for new ooms in manager
Sep 12 00:29:23.635496 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.635482    2558 manager.go:349] Starting recovery of all containers
Sep 12 00:29:23.635760 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:23.635737    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 12 00:29:23.635886 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:23.635868    2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-85.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 12 00:29:23.640116 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.639393    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6swql"
Sep 12 00:29:23.646874 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.646860    2558 manager.go:354] Recovery completed
Sep 12 00:29:23.648028 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:23.648013    2558 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Sep 12 00:29:23.650772 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.650759    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 12 00:29:23.653882 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.653863    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-85.ec2.internal" event="NodeHasSufficientMemory"
Sep 12 00:29:23.653959 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.653895    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-85.ec2.internal" event="NodeHasNoDiskPressure"
Sep 12 00:29:23.653959 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.653906    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-85.ec2.internal" event="NodeHasSufficientPID"
Sep 12 00:29:23.654539 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.654527    2558 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 12 00:29:23.654603 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.654545    2558 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 12 00:29:23.654603 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.654558    2558 state_mem.go:36] "Initialized new in-memory state store"
Sep 12 00:29:23.654603 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.654589    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 12 00:29:23.655719 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.655697    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-85.ec2.internal" event="NodeHasSufficientMemory"
Sep 12 00:29:23.655796 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.655730    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-85.ec2.internal" event="NodeHasNoDiskPressure"
Sep 12 00:29:23.655796 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.655745    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-85.ec2.internal" event="NodeHasSufficientPID"
Sep 12 00:29:23.656162 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.656148    2558 policy_none.go:49] "None policy: Start"
Sep 12 00:29:23.656215 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.656165    2558 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 12 00:29:23.656215 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.656175    2558 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 12 00:29:23.657131 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.657120    2558 policy_none.go:47] "Start"
Sep 12 00:29:23.701699 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.701681    2558 manager.go:337] "Starting Device Plugin manager"
Sep 12 00:29:23.701835 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:23.701735    2558 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 12 00:29:23.701835 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.701752    2558 server.go:87] "Starting device plugin registration server"
Sep 12 00:29:23.702144 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.702132    2558 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 12 00:29:23.702189 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.702150    2558 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 12 00:29:23.702270 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.702256    2558 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 12 00:29:23.702359 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.702347    2558 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 12 00:29:23.702359 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.702358    2558 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 12 00:29:23.702804 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:23.702785    2558 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 12 00:29:23.702897 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:23.702831    2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-85.ec2.internal\" not found"
Sep 12 00:29:23.729687 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.729655    2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 12 00:29:23.731052 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.731036    2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 12 00:29:23.731142 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.731105    2558 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 12 00:29:23.731142 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.731132    2558 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 12 00:29:23.731241 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:23.731177    2558 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 12 00:29:23.733104 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.733064    2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 12 00:29:23.803359 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.803252    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 12 00:29:23.804366 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.804350    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-85.ec2.internal" event="NodeHasSufficientMemory"
Sep 12 00:29:23.804438 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.804382    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-85.ec2.internal" event="NodeHasNoDiskPressure"
Sep 12 00:29:23.804438 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.804392    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-85.ec2.internal" event="NodeHasSufficientPID"
Sep 12 00:29:23.804438 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.804415    2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-85.ec2.internal"
Sep 12 00:29:23.809688 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.809674    2558 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-85.ec2.internal"
Sep 12 00:29:23.809733 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:23.809697    2558 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-85.ec2.internal\": node \"ip-10-0-142-85.ec2.internal\" not found"
Sep 12 00:29:23.822348 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:23.822322    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-85.ec2.internal\" not found"
Sep 12 00:29:23.831315 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.831297    2558 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-85.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-85.ec2.internal"]
Sep 12 00:29:23.831390 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.831359    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 12 00:29:23.832732 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.832716    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-85.ec2.internal" event="NodeHasSufficientMemory"
Sep 12 00:29:23.832805 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.832747    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-85.ec2.internal" event="NodeHasNoDiskPressure"
Sep 12 00:29:23.832805 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.832759    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-85.ec2.internal" event="NodeHasSufficientPID"
Sep 12 00:29:23.833892 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.833881    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 12 00:29:23.833997 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.833975    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-85.ec2.internal"
Sep 12 00:29:23.834126 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.834004    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 12 00:29:23.834510 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.834490    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b08168807839ba0280c207cb77632af-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-85.ec2.internal\" (UID: \"6b08168807839ba0280c207cb77632af\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-85.ec2.internal"
Sep 12 00:29:23.834591 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.834520    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/83844964b8d09ab82b7ac30298c1090d-config\") pod \"kube-apiserver-proxy-ip-10-0-142-85.ec2.internal\" (UID: \"83844964b8d09ab82b7ac30298c1090d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-85.ec2.internal"
Sep 12 00:29:23.834591 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.834569    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6b08168807839ba0280c207cb77632af-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-85.ec2.internal\" (UID: \"6b08168807839ba0280c207cb77632af\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-85.ec2.internal"
Sep 12 00:29:23.834683 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.834635    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-85.ec2.internal" event="NodeHasSufficientMemory"
Sep 12 00:29:23.834683 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.834647    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-85.ec2.internal" event="NodeHasSufficientMemory"
Sep 12 00:29:23.834683 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.834655    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-85.ec2.internal" event="NodeHasNoDiskPressure"
Sep 12 00:29:23.834683 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.834665    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-85.ec2.internal" event="NodeHasNoDiskPressure"
Sep 12 00:29:23.834683 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.834668    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-85.ec2.internal" event="NodeHasSufficientPID"
Sep 12 00:29:23.834683 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.834674    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-85.ec2.internal" event="NodeHasSufficientPID"
Sep 12 00:29:23.835712 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.835696    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-85.ec2.internal"
Sep 12 00:29:23.835788 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.835723    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 12 00:29:23.836399 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.836383    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-85.ec2.internal" event="NodeHasSufficientMemory"
Sep 12 00:29:23.836493 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.836410    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-85.ec2.internal" event="NodeHasNoDiskPressure"
Sep 12 00:29:23.836493 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.836421    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-85.ec2.internal" event="NodeHasSufficientPID"
Sep 12 00:29:23.869358 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:23.869343    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-85.ec2.internal\" not found" node="ip-10-0-142-85.ec2.internal"
Sep 12 00:29:23.873555 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:23.873539    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-85.ec2.internal\" not found" node="ip-10-0-142-85.ec2.internal"
Sep 12 00:29:23.923280 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:23.923250    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-85.ec2.internal\" not found"
Sep 12 00:29:23.935126 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.935080    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6b08168807839ba0280c207cb77632af-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-85.ec2.internal\" (UID: \"6b08168807839ba0280c207cb77632af\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-85.ec2.internal"
Sep 12 00:29:23.935186 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.935138    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b08168807839ba0280c207cb77632af-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-85.ec2.internal\" (UID: \"6b08168807839ba0280c207cb77632af\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-85.ec2.internal"
Sep 12 00:29:23.935186 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.935165    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/83844964b8d09ab82b7ac30298c1090d-config\") pod \"kube-apiserver-proxy-ip-10-0-142-85.ec2.internal\" (UID: \"83844964b8d09ab82b7ac30298c1090d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-85.ec2.internal"
Sep 12 00:29:23.935263 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.935217    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/83844964b8d09ab82b7ac30298c1090d-config\") pod \"kube-apiserver-proxy-ip-10-0-142-85.ec2.internal\" (UID: \"83844964b8d09ab82b7ac30298c1090d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-85.ec2.internal"
Sep 12 00:29:23.935263 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.935217    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6b08168807839ba0280c207cb77632af-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-85.ec2.internal\" (UID: \"6b08168807839ba0280c207cb77632af\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-85.ec2.internal"
Sep 12 00:29:23.935331 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:23.935259    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b08168807839ba0280c207cb77632af-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-85.ec2.internal\" (UID: \"6b08168807839ba0280c207cb77632af\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-85.ec2.internal"
Sep 12 00:29:24.024212 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:24.024165    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-85.ec2.internal\" not found"
Sep 12 00:29:24.124961 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:24.124878    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-85.ec2.internal\" not found"
Sep 12 00:29:24.172396 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:24.172352    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-85.ec2.internal"
Sep 12 00:29:24.176169 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:24.176151    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-85.ec2.internal"
Sep 12 00:29:24.225742 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:24.225708    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-85.ec2.internal\" not found"
Sep 12 00:29:24.326248 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:24.326209    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-85.ec2.internal\" not found"
Sep 12 00:29:24.426777 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:24.426698    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-85.ec2.internal\" not found"
Sep 12 00:29:24.527207 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:24.527171    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-85.ec2.internal\" not found"
Sep 12 00:29:24.576644 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:24.576622    2558 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 12 00:29:24.577014 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:24.576766    2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 12 00:29:24.614287 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:24.614257    2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 12 00:29:24.634043 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:24.634019    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 12 00:29:24.634043 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:24.634035    2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-85.ec2.internal"
Sep 12 00:29:24.642132 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:24.642113    2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 12 00:29:24.643123 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:24.643099    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-11 00:24:24 +0000 UTC" deadline="2028-04-20 10:56:20.932172052 +0000 UTC"
Sep 12 00:29:24.643181 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:24.643125    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14074h26m56.289050701s"
Sep 12 00:29:24.644143 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:24.644129    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 12 00:29:24.645815 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:24.645799    2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-85.ec2.internal"
Sep 12 00:29:24.646059 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:24.646041    2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 12 00:29:24.651701 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:24.651686    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 12 00:29:24.665226 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:24.665207    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6ns55"
Sep 12 00:29:24.671777 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:24.671744    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83844964b8d09ab82b7ac30298c1090d.slice/crio-0dddbf102bc1e68a8e91786409631a7f56344572660c2343f4fde4ce241189f4 WatchSource:0}: Error finding container 0dddbf102bc1e68a8e91786409631a7f56344572660c2343f4fde4ce241189f4: Status 404 returned error can't find the container with id 0dddbf102bc1e68a8e91786409631a7f56344572660c2343f4fde4ce241189f4
Sep 12 00:29:24.671972 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:24.671952    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b08168807839ba0280c207cb77632af.slice/crio-4865f2f93b62c04ad96f028e2929887c878831b9e0b76397869c794bd6cb525c WatchSource:0}: Error finding container 4865f2f93b62c04ad96f028e2929887c878831b9e0b76397869c794bd6cb525c: Status 404 returned error can't find the container with id 4865f2f93b62c04ad96f028e2929887c878831b9e0b76397869c794bd6cb525c
Sep 12 00:29:24.672782 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:24.672762    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6ns55"
Sep 12 00:29:24.676564 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:24.676537    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 12 00:29:24.734841 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:24.734790    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-85.ec2.internal" event={"ID":"83844964b8d09ab82b7ac30298c1090d","Type":"ContainerStarted","Data":"0dddbf102bc1e68a8e91786409631a7f56344572660c2343f4fde4ce241189f4"}
Sep 12 00:29:24.735815 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:24.735794    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-85.ec2.internal" event={"ID":"6b08168807839ba0280c207cb77632af","Type":"ContainerStarted","Data":"4865f2f93b62c04ad96f028e2929887c878831b9e0b76397869c794bd6cb525c"}
Sep 12 00:29:24.878280 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:24.878252    2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 12 00:29:25.612486 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.612453    2558 apiserver.go:52] "Watching apiserver"
Sep 12 00:29:25.622668 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.622641    2558 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 12 00:29:25.623068 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.623035    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9pms","openshift-multus/multus-mztg9","openshift-network-operator/iptables-alerter-llf45","openshift-ovn-kubernetes/ovnkube-node-cmk6p","openshift-cluster-node-tuning-operator/tuned-kltnf","openshift-dns/node-resolver-gs67t","openshift-image-registry/node-ca-fd27q","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-85.ec2.internal","openshift-multus/multus-additional-cni-plugins-m6mrh","openshift-multus/network-metrics-daemon-7tkvx","openshift-network-diagnostics/network-check-target-f6t44","kube-system/global-pull-secret-syncer-b2hjw","kube-system/konnectivity-agent-9dh42","kube-system/kube-apiserver-proxy-ip-10-0-142-85.ec2.internal"]
Sep 12 00:29:25.624588 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.624569    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fd27q"
Sep 12 00:29:25.625934 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.625911    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9dh42"
Sep 12 00:29:25.627191 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.627145    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-llf45"
Sep 12 00:29:25.627299 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.627206    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 12 00:29:25.627359 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.627304    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 12 00:29:25.627414 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.627346    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-6mc6h\""
Sep 12 00:29:25.627544 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.627528    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 12 00:29:25.628402 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.628380    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-65822\""
Sep 12 00:29:25.628501 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.628484    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 12 00:29:25.628762 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.628726    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 12 00:29:25.628858 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.628810    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.630217 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.629467    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 12 00:29:25.630217 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.629715    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 12 00:29:25.630217 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.629792    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 12 00:29:25.630217 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.629805    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.630217 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.629936    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-t7zbh\""
Sep 12 00:29:25.630904 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.630885    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-rkjms\""
Sep 12 00:29:25.631159 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.631137    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gs67t"
Sep 12 00:29:25.631705 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.631282    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 12 00:29:25.631705 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.631310    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 12 00:29:25.631705 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.631313    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 12 00:29:25.631705 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.631550    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 12 00:29:25.631705 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.631613    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 12 00:29:25.632853 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.632521    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-m6mrh"
Sep 12 00:29:25.633660 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.633193    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 12 00:29:25.633660 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.633202    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6bw57\""
Sep 12 00:29:25.633660 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.633420    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 12 00:29:25.633660 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.633438    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 12 00:29:25.633870 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.633738    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hpjr4\""
Sep 12 00:29:25.633870 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.633761    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 12 00:29:25.634051 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.634026    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 12 00:29:25.634914 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.634692    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9pms"
Sep 12 00:29:25.635829 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.635783    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 12 00:29:25.635829 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.635788    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 12 00:29:25.635987 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.635829    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 12 00:29:25.635987 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.635955    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-mf2s8\""
Sep 12 00:29:25.636081 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.636033    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 12 00:29:25.636081 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.636056    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 12 00:29:25.636417 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.636386    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.636899 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.636876    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 12 00:29:25.637354 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.637334    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 12 00:29:25.637467 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.637448    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-m9rdj\""
Sep 12 00:29:25.637467 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.637463    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 12 00:29:25.637984 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.637966    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:29:25.638176 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:25.638128    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tkvx" podUID="746054e5-684e-4c55-824b-6fdde562eb02"
Sep 12 00:29:25.638592 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.638571    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 12 00:29:25.638849 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.638831    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xgzzl\""
Sep 12 00:29:25.639799 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.639777    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f6t44"
Sep 12 00:29:25.639903 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:25.639879    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f6t44" podUID="550df110-4473-4487-b318-d72d9babcaa3"
Sep 12 00:29:25.642470 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.642450    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:29:25.642631 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:25.642600    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2hjw" podUID="ed1e49a7-a9ae-4e55-8a01-c88326238a56"
Sep 12 00:29:25.644238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.644085    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-host-var-lib-cni-multus\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.644238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.644149    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-var-lib-openvswitch\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.644238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.644181    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-etc-openvswitch\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.644238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.644213    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-host-cni-bin\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.644495 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.644242    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-host-cni-netd\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.644495 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.644271    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-etc-modprobe-d\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.644495 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.644299    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-cnibin\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.644495 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.644356    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-run-systemd\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.644495 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.644386    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/45051c76-e610-4f10-bb3f-521031f82611-ovnkube-config\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.644495 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.644420    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbxqn\" (UniqueName: \"kubernetes.io/projected/34aa479e-166d-4d92-8909-a58f6e2649ef-kube-api-access-zbxqn\") pod \"node-resolver-gs67t\" (UID: \"34aa479e-166d-4d92-8909-a58f6e2649ef\") " pod="openshift-dns/node-resolver-gs67t"
Sep 12 00:29:25.644495 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.644463    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cc3d5554-d660-453c-bc3e-4d3ed765400e-konnectivity-ca\") pod \"konnectivity-agent-9dh42\" (UID: \"cc3d5554-d660-453c-bc3e-4d3ed765400e\") " pod="kube-system/konnectivity-agent-9dh42"
Sep 12 00:29:25.644495 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.644495    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-run-openvswitch\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.644850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.644522    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-run-ovn\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.644850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.644571    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-etc-sysconfig\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.644850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.644605    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-var-lib-kubelet\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.644850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.644637    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ca5a34af-39bd-486b-8293-387ff1f949e2-system-cni-dir\") pod \"multus-additional-cni-plugins-m6mrh\" (UID: \"ca5a34af-39bd-486b-8293-387ff1f949e2\") " pod="openshift-multus/multus-additional-cni-plugins-m6mrh"
Sep 12 00:29:25.644850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.644676    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgtc6\" (UniqueName: \"kubernetes.io/projected/c17577fa-fd1b-45a2-9d4d-6be7144ba759-kube-api-access-vgtc6\") pod \"node-ca-fd27q\" (UID: \"c17577fa-fd1b-45a2-9d4d-6be7144ba759\") " pod="openshift-image-registry/node-ca-fd27q"
Sep 12 00:29:25.644850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.644707    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-host-var-lib-kubelet\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.644850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.644736    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-hostroot\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.644850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.644771    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-host-run-netns\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.644850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.644802    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/45051c76-e610-4f10-bb3f-521031f82611-ovnkube-script-lib\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.644850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.644835    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-sys\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.645246 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.644867    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-lib-modules\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.645246 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.644895    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/28b661ed-bfe5-4fbe-8bf0-b2de036449ac-registration-dir\") pod \"aws-ebs-csi-driver-node-r9pms\" (UID: \"28b661ed-bfe5-4fbe-8bf0-b2de036449ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9pms"
Sep 12 00:29:25.645246 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.644930    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7cc8a470-64c8-4446-9e7a-994fb6309a10-iptables-alerter-script\") pod \"iptables-alerter-llf45\" (UID: \"7cc8a470-64c8-4446-9e7a-994fb6309a10\") " pod="openshift-network-operator/iptables-alerter-llf45"
Sep 12 00:29:25.645246 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.644969    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ca5a34af-39bd-486b-8293-387ff1f949e2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m6mrh\" (UID: \"ca5a34af-39bd-486b-8293-387ff1f949e2\") " pod="openshift-multus/multus-additional-cni-plugins-m6mrh"
Sep 12 00:29:25.645246 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.645027    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ca5a34af-39bd-486b-8293-387ff1f949e2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m6mrh\" (UID: \"ca5a34af-39bd-486b-8293-387ff1f949e2\") " pod="openshift-multus/multus-additional-cni-plugins-m6mrh"
Sep 12 00:29:25.645246 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.645061    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfqvk\" (UniqueName: \"kubernetes.io/projected/4d786af9-7b4e-4fd7-85bd-8e67570b009d-kube-api-access-bfqvk\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.645246 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.645101    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cc3d5554-d660-453c-bc3e-4d3ed765400e-agent-certs\") pod \"konnectivity-agent-9dh42\" (UID: \"cc3d5554-d660-453c-bc3e-4d3ed765400e\") " pod="kube-system/konnectivity-agent-9dh42"
Sep 12 00:29:25.645246 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.645193    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-etc-sysctl-conf\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.645246 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.645233    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-etc-systemd\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.645614 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.645267    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-etc-tuned\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.645614 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.645295    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-multus-conf-dir\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.645614 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.645325    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-run\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.645614 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.645364    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-multus-socket-dir-parent\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.645614 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.645401    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-host-run-ovn-kubernetes\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.645614 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.645442    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxlb4\" (UniqueName: \"kubernetes.io/projected/45051c76-e610-4f10-bb3f-521031f82611-kube-api-access-lxlb4\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.645614 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.645475    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/28b661ed-bfe5-4fbe-8bf0-b2de036449ac-device-dir\") pod \"aws-ebs-csi-driver-node-r9pms\" (UID: \"28b661ed-bfe5-4fbe-8bf0-b2de036449ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9pms"
Sep 12 00:29:25.645614 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.645512    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/28b661ed-bfe5-4fbe-8bf0-b2de036449ac-etc-selinux\") pod \"aws-ebs-csi-driver-node-r9pms\" (UID: \"28b661ed-bfe5-4fbe-8bf0-b2de036449ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9pms"
Sep 12 00:29:25.645614 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.645540    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-system-cni-dir\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.645614 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.645571    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-systemd-units\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.645614 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.645608    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.645968 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.645648    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ca5a34af-39bd-486b-8293-387ff1f949e2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m6mrh\" (UID: \"ca5a34af-39bd-486b-8293-387ff1f949e2\") " pod="openshift-multus/multus-additional-cni-plugins-m6mrh"
Sep 12 00:29:25.645968 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.645696    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/28b661ed-bfe5-4fbe-8bf0-b2de036449ac-socket-dir\") pod \"aws-ebs-csi-driver-node-r9pms\" (UID: \"28b661ed-bfe5-4fbe-8bf0-b2de036449ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9pms"
Sep 12 00:29:25.645968 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.645729    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xtrc\" (UniqueName: \"kubernetes.io/projected/7cc8a470-64c8-4446-9e7a-994fb6309a10-kube-api-access-4xtrc\") pod \"iptables-alerter-llf45\" (UID: \"7cc8a470-64c8-4446-9e7a-994fb6309a10\") " pod="openshift-network-operator/iptables-alerter-llf45"
Sep 12 00:29:25.645968 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.645764    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-etc-sysctl-d\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.645968 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.645793    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-host\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.645968 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.645824    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34aa479e-166d-4d92-8909-a58f6e2649ef-tmp-dir\") pod \"node-resolver-gs67t\" (UID: \"34aa479e-166d-4d92-8909-a58f6e2649ef\") " pod="openshift-dns/node-resolver-gs67t"
Sep 12 00:29:25.645968 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.645850    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ca5a34af-39bd-486b-8293-387ff1f949e2-cnibin\") pod \"multus-additional-cni-plugins-m6mrh\" (UID: \"ca5a34af-39bd-486b-8293-387ff1f949e2\") " pod="openshift-multus/multus-additional-cni-plugins-m6mrh"
Sep 12 00:29:25.645968 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.645880    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c17577fa-fd1b-45a2-9d4d-6be7144ba759-serviceca\") pod \"node-ca-fd27q\" (UID: \"c17577fa-fd1b-45a2-9d4d-6be7144ba759\") " pod="openshift-image-registry/node-ca-fd27q"
Sep 12 00:29:25.646306 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.645962    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-multus-cni-dir\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.646306 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.646013    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4d786af9-7b4e-4fd7-85bd-8e67570b009d-cni-binary-copy\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.646306 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.646079    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-host-run-netns\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.647457 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.647395    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-etc-kubernetes\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.647457 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.647443    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-log-socket\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.647825 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.647476    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/45051c76-e610-4f10-bb3f-521031f82611-ovn-node-metrics-cert\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.647825 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.647511    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/45051c76-e610-4f10-bb3f-521031f82611-env-overrides\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.647825 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.647553    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-etc-kubernetes\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.647825 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.647585    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xvrb\" (UniqueName: \"kubernetes.io/projected/28b661ed-bfe5-4fbe-8bf0-b2de036449ac-kube-api-access-5xvrb\") pod \"aws-ebs-csi-driver-node-r9pms\" (UID: \"28b661ed-bfe5-4fbe-8bf0-b2de036449ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9pms"
Sep 12 00:29:25.647825 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.647618    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-host-var-lib-cni-bin\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.647825 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.647663    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-host-run-multus-certs\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.647825 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.647715    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-host-kubelet\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.647825 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.647751    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-host-slash\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.647825 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.647771    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-node-log\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.647825 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.647821    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ca5a34af-39bd-486b-8293-387ff1f949e2-cni-binary-copy\") pod \"multus-additional-cni-plugins-m6mrh\" (UID: \"ca5a34af-39bd-486b-8293-387ff1f949e2\") " pod="openshift-multus/multus-additional-cni-plugins-m6mrh"
Sep 12 00:29:25.648367 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.647863    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4d786af9-7b4e-4fd7-85bd-8e67570b009d-multus-daemon-config\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.648367 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.647898    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98rxv\" (UniqueName: \"kubernetes.io/projected/ca5a34af-39bd-486b-8293-387ff1f949e2-kube-api-access-98rxv\") pod \"multus-additional-cni-plugins-m6mrh\" (UID: \"ca5a34af-39bd-486b-8293-387ff1f949e2\") " pod="openshift-multus/multus-additional-cni-plugins-m6mrh"
Sep 12 00:29:25.648367 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.648014    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/28b661ed-bfe5-4fbe-8bf0-b2de036449ac-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r9pms\" (UID: \"28b661ed-bfe5-4fbe-8bf0-b2de036449ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9pms"
Sep 12 00:29:25.648367 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.648074    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/28b661ed-bfe5-4fbe-8bf0-b2de036449ac-sys-fs\") pod \"aws-ebs-csi-driver-node-r9pms\" (UID: \"28b661ed-bfe5-4fbe-8bf0-b2de036449ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9pms"
Sep 12 00:29:25.648367 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.648114    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c17577fa-fd1b-45a2-9d4d-6be7144ba759-host\") pod \"node-ca-fd27q\" (UID: \"c17577fa-fd1b-45a2-9d4d-6be7144ba759\") " pod="openshift-image-registry/node-ca-fd27q"
Sep 12 00:29:25.648367 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.648151    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-host-run-k8s-cni-cncf-io\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.648630 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.648581    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7cc8a470-64c8-4446-9e7a-994fb6309a10-host-slash\") pod \"iptables-alerter-llf45\" (UID: \"7cc8a470-64c8-4446-9e7a-994fb6309a10\") " pod="openshift-network-operator/iptables-alerter-llf45"
Sep 12 00:29:25.648630 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.648617    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-tmp\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.648737 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.648647    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-885zl\" (UniqueName: \"kubernetes.io/projected/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-kube-api-access-885zl\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.648737 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.648688    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/34aa479e-166d-4d92-8909-a58f6e2649ef-hosts-file\") pod \"node-resolver-gs67t\" (UID: \"34aa479e-166d-4d92-8909-a58f6e2649ef\") " pod="openshift-dns/node-resolver-gs67t"
Sep 12 00:29:25.675341 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.675288    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-11 00:24:25 +0000 UTC" deadline="2028-04-07 11:20:53.33751061 +0000 UTC"
Sep 12 00:29:25.675452 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.675342    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13762h51m27.66217333s"
Sep 12 00:29:25.735206 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.735178    2558 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 12 00:29:25.749178 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749145    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-etc-sysctl-conf\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.749407 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749193    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-etc-systemd\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.749407 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749277    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-etc-systemd\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.749407 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749317    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-etc-tuned\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.749407 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749360    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-multus-conf-dir\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.749407 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749391    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-run\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.749407 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749401    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-multus-conf-dir\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.749735 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749396    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-etc-sysctl-conf\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.749735 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749419    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwrlm\" (UniqueName: \"kubernetes.io/projected/550df110-4473-4487-b318-d72d9babcaa3-kube-api-access-kwrlm\") pod \"network-check-target-f6t44\" (UID: \"550df110-4473-4487-b318-d72d9babcaa3\") " pod="openshift-network-diagnostics/network-check-target-f6t44"
Sep 12 00:29:25.749735 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749452    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-run\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.749735 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749450    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-multus-socket-dir-parent\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.749735 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749500    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-host-run-ovn-kubernetes\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.749735 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749504    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-multus-socket-dir-parent\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.749735 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749528    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxlb4\" (UniqueName: \"kubernetes.io/projected/45051c76-e610-4f10-bb3f-521031f82611-kube-api-access-lxlb4\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.749735 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749539    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-host-run-ovn-kubernetes\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.749735 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749555    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/28b661ed-bfe5-4fbe-8bf0-b2de036449ac-device-dir\") pod \"aws-ebs-csi-driver-node-r9pms\" (UID: \"28b661ed-bfe5-4fbe-8bf0-b2de036449ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9pms"
Sep 12 00:29:25.749735 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749602    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/28b661ed-bfe5-4fbe-8bf0-b2de036449ac-etc-selinux\") pod \"aws-ebs-csi-driver-node-r9pms\" (UID: \"28b661ed-bfe5-4fbe-8bf0-b2de036449ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9pms"
Sep 12 00:29:25.749735 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749641    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ed1e49a7-a9ae-4e55-8a01-c88326238a56-kubelet-config\") pod \"global-pull-secret-syncer-b2hjw\" (UID: \"ed1e49a7-a9ae-4e55-8a01-c88326238a56\") " pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:29:25.749735 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749655    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/28b661ed-bfe5-4fbe-8bf0-b2de036449ac-etc-selinux\") pod \"aws-ebs-csi-driver-node-r9pms\" (UID: \"28b661ed-bfe5-4fbe-8bf0-b2de036449ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9pms"
Sep 12 00:29:25.749735 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749674    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-system-cni-dir\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.749735 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749702    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-systemd-units\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.749735 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749728    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.750469 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749753    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ca5a34af-39bd-486b-8293-387ff1f949e2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m6mrh\" (UID: \"ca5a34af-39bd-486b-8293-387ff1f949e2\") " pod="openshift-multus/multus-additional-cni-plugins-m6mrh"
Sep 12 00:29:25.750469 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749768    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-systemd-units\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.750469 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749778    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/28b661ed-bfe5-4fbe-8bf0-b2de036449ac-socket-dir\") pod \"aws-ebs-csi-driver-node-r9pms\" (UID: \"28b661ed-bfe5-4fbe-8bf0-b2de036449ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9pms"
Sep 12 00:29:25.750469 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749804    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xtrc\" (UniqueName: \"kubernetes.io/projected/7cc8a470-64c8-4446-9e7a-994fb6309a10-kube-api-access-4xtrc\") pod \"iptables-alerter-llf45\" (UID: \"7cc8a470-64c8-4446-9e7a-994fb6309a10\") " pod="openshift-network-operator/iptables-alerter-llf45"
Sep 12 00:29:25.750469 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749808    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/28b661ed-bfe5-4fbe-8bf0-b2de036449ac-device-dir\") pod \"aws-ebs-csi-driver-node-r9pms\" (UID: \"28b661ed-bfe5-4fbe-8bf0-b2de036449ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9pms"
Sep 12 00:29:25.750469 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749832    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.750469 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749831    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-etc-sysctl-d\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.750469 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749876    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-system-cni-dir\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.750469 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749876    2558 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 12 00:29:25.750469 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749895    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-host\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.750469 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749931    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-etc-sysctl-d\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.750469 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749939    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34aa479e-166d-4d92-8909-a58f6e2649ef-tmp-dir\") pod \"node-resolver-gs67t\" (UID: \"34aa479e-166d-4d92-8909-a58f6e2649ef\") " pod="openshift-dns/node-resolver-gs67t"
Sep 12 00:29:25.750469 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749973    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ca5a34af-39bd-486b-8293-387ff1f949e2-cnibin\") pod \"multus-additional-cni-plugins-m6mrh\" (UID: \"ca5a34af-39bd-486b-8293-387ff1f949e2\") " pod="openshift-multus/multus-additional-cni-plugins-m6mrh"
Sep 12 00:29:25.750469 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.749949    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/28b661ed-bfe5-4fbe-8bf0-b2de036449ac-socket-dir\") pod \"aws-ebs-csi-driver-node-r9pms\" (UID: \"28b661ed-bfe5-4fbe-8bf0-b2de036449ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9pms"
Sep 12 00:29:25.750469 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.750001    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c17577fa-fd1b-45a2-9d4d-6be7144ba759-serviceca\") pod \"node-ca-fd27q\" (UID: \"c17577fa-fd1b-45a2-9d4d-6be7144ba759\") " pod="openshift-image-registry/node-ca-fd27q"
Sep 12 00:29:25.750469 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.750029    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-multus-cni-dir\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.750469 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.750075    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4d786af9-7b4e-4fd7-85bd-8e67570b009d-cni-binary-copy\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.750469 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.750115    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-host-run-netns\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.751434 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.750175    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-etc-kubernetes\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.751434 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.750187    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-host\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.751434 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.750203    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-log-socket\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.751434 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.750247    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-log-socket\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.751434 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.750254    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/45051c76-e610-4f10-bb3f-521031f82611-ovn-node-metrics-cert\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.751434 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.750281    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/45051c76-e610-4f10-bb3f-521031f82611-env-overrides\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.751434 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.750292    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ca5a34af-39bd-486b-8293-387ff1f949e2-cnibin\") pod \"multus-additional-cni-plugins-m6mrh\" (UID: \"ca5a34af-39bd-486b-8293-387ff1f949e2\") " pod="openshift-multus/multus-additional-cni-plugins-m6mrh"
Sep 12 00:29:25.751434 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.750296    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34aa479e-166d-4d92-8909-a58f6e2649ef-tmp-dir\") pod \"node-resolver-gs67t\" (UID: \"34aa479e-166d-4d92-8909-a58f6e2649ef\") " pod="openshift-dns/node-resolver-gs67t"
Sep 12 00:29:25.751434 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.750307    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-etc-kubernetes\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.751434 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.750393    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xvrb\" (UniqueName: \"kubernetes.io/projected/28b661ed-bfe5-4fbe-8bf0-b2de036449ac-kube-api-access-5xvrb\") pod \"aws-ebs-csi-driver-node-r9pms\" (UID: \"28b661ed-bfe5-4fbe-8bf0-b2de036449ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9pms"
Sep 12 00:29:25.751434 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.750427    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-host-var-lib-cni-bin\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.751434 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.750459    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-host-run-multus-certs\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.751434 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.750522    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-host-run-multus-certs\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.751434 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.750543    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ca5a34af-39bd-486b-8293-387ff1f949e2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m6mrh\" (UID: \"ca5a34af-39bd-486b-8293-387ff1f949e2\") " pod="openshift-multus/multus-additional-cni-plugins-m6mrh"
Sep 12 00:29:25.751434 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.750587    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-multus-cni-dir\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.751434 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.750655    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-host-kubelet\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.751434 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.750769    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-host-var-lib-cni-bin\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.751434 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.750776    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-etc-kubernetes\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.752387 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.751008    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/45051c76-e610-4f10-bb3f-521031f82611-env-overrides\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.752387 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.751064    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-host-slash\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.752387 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.751088    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-host-kubelet\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.752387 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.751093    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-node-log\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.752387 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.751131    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-node-log\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.752387 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.751193    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ca5a34af-39bd-486b-8293-387ff1f949e2-cni-binary-copy\") pod \"multus-additional-cni-plugins-m6mrh\" (UID: \"ca5a34af-39bd-486b-8293-387ff1f949e2\") " pod="openshift-multus/multus-additional-cni-plugins-m6mrh"
Sep 12 00:29:25.752387 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.751225    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4d786af9-7b4e-4fd7-85bd-8e67570b009d-multus-daemon-config\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.752387 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.751270    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98rxv\" (UniqueName: \"kubernetes.io/projected/ca5a34af-39bd-486b-8293-387ff1f949e2-kube-api-access-98rxv\") pod \"multus-additional-cni-plugins-m6mrh\" (UID: \"ca5a34af-39bd-486b-8293-387ff1f949e2\") " pod="openshift-multus/multus-additional-cni-plugins-m6mrh"
Sep 12 00:29:25.752387 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.751354    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c17577fa-fd1b-45a2-9d4d-6be7144ba759-serviceca\") pod \"node-ca-fd27q\" (UID: \"c17577fa-fd1b-45a2-9d4d-6be7144ba759\") " pod="openshift-image-registry/node-ca-fd27q"
Sep 12 00:29:25.752387 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.751369    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/28b661ed-bfe5-4fbe-8bf0-b2de036449ac-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r9pms\" (UID: \"28b661ed-bfe5-4fbe-8bf0-b2de036449ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9pms"
Sep 12 00:29:25.752387 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.751396    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/28b661ed-bfe5-4fbe-8bf0-b2de036449ac-sys-fs\") pod \"aws-ebs-csi-driver-node-r9pms\" (UID: \"28b661ed-bfe5-4fbe-8bf0-b2de036449ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9pms"
Sep 12 00:29:25.752387 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.751419    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-host-slash\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.752387 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.751421    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c17577fa-fd1b-45a2-9d4d-6be7144ba759-host\") pod \"node-ca-fd27q\" (UID: \"c17577fa-fd1b-45a2-9d4d-6be7144ba759\") " pod="openshift-image-registry/node-ca-fd27q"
Sep 12 00:29:25.752387 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.751427    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4d786af9-7b4e-4fd7-85bd-8e67570b009d-cni-binary-copy\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.752387 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.751463    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-host-run-k8s-cni-cncf-io\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.752387 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.751500    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7cc8a470-64c8-4446-9e7a-994fb6309a10-host-slash\") pod \"iptables-alerter-llf45\" (UID: \"7cc8a470-64c8-4446-9e7a-994fb6309a10\") " pod="openshift-network-operator/iptables-alerter-llf45"
Sep 12 00:29:25.752387 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.751518    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/28b661ed-bfe5-4fbe-8bf0-b2de036449ac-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r9pms\" (UID: \"28b661ed-bfe5-4fbe-8bf0-b2de036449ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9pms"
Sep 12 00:29:25.752387 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.751558    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-tmp\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.753377 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.751584    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-885zl\" (UniqueName: \"kubernetes.io/projected/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-kube-api-access-885zl\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.753377 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.751611    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/34aa479e-166d-4d92-8909-a58f6e2649ef-hosts-file\") pod \"node-resolver-gs67t\" (UID: \"34aa479e-166d-4d92-8909-a58f6e2649ef\") " pod="openshift-dns/node-resolver-gs67t"
Sep 12 00:29:25.753377 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.751662    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ca5a34af-39bd-486b-8293-387ff1f949e2-cni-binary-copy\") pod \"multus-additional-cni-plugins-m6mrh\" (UID: \"ca5a34af-39bd-486b-8293-387ff1f949e2\") " pod="openshift-multus/multus-additional-cni-plugins-m6mrh"
Sep 12 00:29:25.753377 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.751706    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7cc8a470-64c8-4446-9e7a-994fb6309a10-host-slash\") pod \"iptables-alerter-llf45\" (UID: \"7cc8a470-64c8-4446-9e7a-994fb6309a10\") " pod="openshift-network-operator/iptables-alerter-llf45"
Sep 12 00:29:25.753377 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.751718    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcqxx\" (UniqueName: \"kubernetes.io/projected/746054e5-684e-4c55-824b-6fdde562eb02-kube-api-access-zcqxx\") pod \"network-metrics-daemon-7tkvx\" (UID: \"746054e5-684e-4c55-824b-6fdde562eb02\") " pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:29:25.753377 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.751725    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-etc-kubernetes\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.753377 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.751774    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/34aa479e-166d-4d92-8909-a58f6e2649ef-hosts-file\") pod \"node-resolver-gs67t\" (UID: \"34aa479e-166d-4d92-8909-a58f6e2649ef\") " pod="openshift-dns/node-resolver-gs67t"
Sep 12 00:29:25.753377 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.751803    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-host-var-lib-cni-multus\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.753377 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.751838    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-var-lib-openvswitch\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.753377 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.751880    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-etc-openvswitch\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.753377 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.751900    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-host-cni-bin\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.753377 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.751966    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-host-cni-netd\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.753377 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.751987    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4d786af9-7b4e-4fd7-85bd-8e67570b009d-multus-daemon-config\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.753377 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.752024    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c17577fa-fd1b-45a2-9d4d-6be7144ba759-host\") pod \"node-ca-fd27q\" (UID: \"c17577fa-fd1b-45a2-9d4d-6be7144ba759\") " pod="openshift-image-registry/node-ca-fd27q"
Sep 12 00:29:25.753377 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.752082    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/28b661ed-bfe5-4fbe-8bf0-b2de036449ac-sys-fs\") pod \"aws-ebs-csi-driver-node-r9pms\" (UID: \"28b661ed-bfe5-4fbe-8bf0-b2de036449ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9pms"
Sep 12 00:29:25.753377 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.752039    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-host-run-netns\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.753377 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.751796    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-host-run-k8s-cni-cncf-io\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.753377 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.752016    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-etc-modprobe-d\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.754295 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.752140    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ed1e49a7-a9ae-4e55-8a01-c88326238a56-dbus\") pod \"global-pull-secret-syncer-b2hjw\" (UID: \"ed1e49a7-a9ae-4e55-8a01-c88326238a56\") " pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:29:25.754295 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.752183    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-etc-modprobe-d\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.754295 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.752189    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-cnibin\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.754295 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.752182    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-host-cni-netd\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.754295 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.752226    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-var-lib-openvswitch\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.754295 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.752235    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-run-systemd\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.754295 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.752240    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-host-cni-bin\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.754295 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.752263    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-etc-openvswitch\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.754295 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.752265    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/45051c76-e610-4f10-bb3f-521031f82611-ovnkube-config\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.754295 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.752318    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-run-systemd\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.754295 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.752413    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-cnibin\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.754295 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.752480    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbxqn\" (UniqueName: \"kubernetes.io/projected/34aa479e-166d-4d92-8909-a58f6e2649ef-kube-api-access-zbxqn\") pod \"node-resolver-gs67t\" (UID: \"34aa479e-166d-4d92-8909-a58f6e2649ef\") " pod="openshift-dns/node-resolver-gs67t"
Sep 12 00:29:25.754295 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.752538    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cc3d5554-d660-453c-bc3e-4d3ed765400e-konnectivity-ca\") pod \"konnectivity-agent-9dh42\" (UID: \"cc3d5554-d660-453c-bc3e-4d3ed765400e\") " pod="kube-system/konnectivity-agent-9dh42"
Sep 12 00:29:25.754295 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.752565    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-run-openvswitch\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.754295 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.752606    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-run-ovn\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.754295 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.752648    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-etc-sysconfig\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.754295 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.752674    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-var-lib-kubelet\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.754295 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.752710    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ca5a34af-39bd-486b-8293-387ff1f949e2-system-cni-dir\") pod \"multus-additional-cni-plugins-m6mrh\" (UID: \"ca5a34af-39bd-486b-8293-387ff1f949e2\") " pod="openshift-multus/multus-additional-cni-plugins-m6mrh"
Sep 12 00:29:25.755246 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.752751    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ed1e49a7-a9ae-4e55-8a01-c88326238a56-original-pull-secret\") pod \"global-pull-secret-syncer-b2hjw\" (UID: \"ed1e49a7-a9ae-4e55-8a01-c88326238a56\") " pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:29:25.755246 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.752783    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgtc6\" (UniqueName: \"kubernetes.io/projected/c17577fa-fd1b-45a2-9d4d-6be7144ba759-kube-api-access-vgtc6\") pod \"node-ca-fd27q\" (UID: \"c17577fa-fd1b-45a2-9d4d-6be7144ba759\") " pod="openshift-image-registry/node-ca-fd27q"
Sep 12 00:29:25.755246 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.752793    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/45051c76-e610-4f10-bb3f-521031f82611-ovnkube-config\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.755246 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.752817    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-host-var-lib-kubelet\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.755246 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.752851    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-hostroot\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.755246 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.752862    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-etc-sysconfig\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.755246 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.752895    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-host-run-netns\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.755246 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.752922    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/45051c76-e610-4f10-bb3f-521031f82611-ovnkube-script-lib\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.755246 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.752945    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-sys\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.755246 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.752952    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-var-lib-kubelet\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.755246 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.752968    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-lib-modules\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.755246 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.753002    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-run-openvswitch\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.755246 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.753015    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/28b661ed-bfe5-4fbe-8bf0-b2de036449ac-registration-dir\") pod \"aws-ebs-csi-driver-node-r9pms\" (UID: \"28b661ed-bfe5-4fbe-8bf0-b2de036449ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9pms"
Sep 12 00:29:25.755246 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.753054    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-host-run-netns\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.755246 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.753085    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/28b661ed-bfe5-4fbe-8bf0-b2de036449ac-registration-dir\") pod \"aws-ebs-csi-driver-node-r9pms\" (UID: \"28b661ed-bfe5-4fbe-8bf0-b2de036449ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9pms"
Sep 12 00:29:25.755246 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.753085    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7cc8a470-64c8-4446-9e7a-994fb6309a10-iptables-alerter-script\") pod \"iptables-alerter-llf45\" (UID: \"7cc8a470-64c8-4446-9e7a-994fb6309a10\") " pod="openshift-network-operator/iptables-alerter-llf45"
Sep 12 00:29:25.755246 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.752893    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ca5a34af-39bd-486b-8293-387ff1f949e2-system-cni-dir\") pod \"multus-additional-cni-plugins-m6mrh\" (UID: \"ca5a34af-39bd-486b-8293-387ff1f949e2\") " pod="openshift-multus/multus-additional-cni-plugins-m6mrh"
Sep 12 00:29:25.756238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.753235    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ca5a34af-39bd-486b-8293-387ff1f949e2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m6mrh\" (UID: \"ca5a34af-39bd-486b-8293-387ff1f949e2\") " pod="openshift-multus/multus-additional-cni-plugins-m6mrh"
Sep 12 00:29:25.756238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.753262    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ca5a34af-39bd-486b-8293-387ff1f949e2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m6mrh\" (UID: \"ca5a34af-39bd-486b-8293-387ff1f949e2\") " pod="openshift-multus/multus-additional-cni-plugins-m6mrh"
Sep 12 00:29:25.756238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.753314    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/746054e5-684e-4c55-824b-6fdde562eb02-metrics-certs\") pod \"network-metrics-daemon-7tkvx\" (UID: \"746054e5-684e-4c55-824b-6fdde562eb02\") " pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:29:25.756238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.753359    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfqvk\" (UniqueName: \"kubernetes.io/projected/4d786af9-7b4e-4fd7-85bd-8e67570b009d-kube-api-access-bfqvk\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.756238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.753384    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cc3d5554-d660-453c-bc3e-4d3ed765400e-agent-certs\") pod \"konnectivity-agent-9dh42\" (UID: \"cc3d5554-d660-453c-bc3e-4d3ed765400e\") " pod="kube-system/konnectivity-agent-9dh42"
Sep 12 00:29:25.756238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.753440    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cc3d5554-d660-453c-bc3e-4d3ed765400e-konnectivity-ca\") pod \"konnectivity-agent-9dh42\" (UID: \"cc3d5554-d660-453c-bc3e-4d3ed765400e\") " pod="kube-system/konnectivity-agent-9dh42"
Sep 12 00:29:25.756238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.753698    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7cc8a470-64c8-4446-9e7a-994fb6309a10-iptables-alerter-script\") pod \"iptables-alerter-llf45\" (UID: \"7cc8a470-64c8-4446-9e7a-994fb6309a10\") " pod="openshift-network-operator/iptables-alerter-llf45"
Sep 12 00:29:25.756238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.753883    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-host-var-lib-cni-multus\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.756238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.753971    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/45051c76-e610-4f10-bb3f-521031f82611-run-ovn\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.756238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.753955    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-lib-modules\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.756238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.754017    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-sys\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.756238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.754064    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-hostroot\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.756238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.754149    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-etc-tuned\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.756238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.754234    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ca5a34af-39bd-486b-8293-387ff1f949e2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m6mrh\" (UID: \"ca5a34af-39bd-486b-8293-387ff1f949e2\") " pod="openshift-multus/multus-additional-cni-plugins-m6mrh"
Sep 12 00:29:25.756238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.754279    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d786af9-7b4e-4fd7-85bd-8e67570b009d-host-var-lib-kubelet\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.756238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.754430    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-tmp\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.756238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.754510    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ca5a34af-39bd-486b-8293-387ff1f949e2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m6mrh\" (UID: \"ca5a34af-39bd-486b-8293-387ff1f949e2\") " pod="openshift-multus/multus-additional-cni-plugins-m6mrh"
Sep 12 00:29:25.757123 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.754831    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/45051c76-e610-4f10-bb3f-521031f82611-ovn-node-metrics-cert\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.757123 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.755378    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/45051c76-e610-4f10-bb3f-521031f82611-ovnkube-script-lib\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.757123 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.756976    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cc3d5554-d660-453c-bc3e-4d3ed765400e-agent-certs\") pod \"konnectivity-agent-9dh42\" (UID: \"cc3d5554-d660-453c-bc3e-4d3ed765400e\") " pod="kube-system/konnectivity-agent-9dh42"
Sep 12 00:29:25.758953 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.758314    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xtrc\" (UniqueName: \"kubernetes.io/projected/7cc8a470-64c8-4446-9e7a-994fb6309a10-kube-api-access-4xtrc\") pod \"iptables-alerter-llf45\" (UID: \"7cc8a470-64c8-4446-9e7a-994fb6309a10\") " pod="openshift-network-operator/iptables-alerter-llf45"
Sep 12 00:29:25.758953 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.758642    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxlb4\" (UniqueName: \"kubernetes.io/projected/45051c76-e610-4f10-bb3f-521031f82611-kube-api-access-lxlb4\") pod \"ovnkube-node-cmk6p\" (UID: \"45051c76-e610-4f10-bb3f-521031f82611\") " pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.758953 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.758791    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xvrb\" (UniqueName: \"kubernetes.io/projected/28b661ed-bfe5-4fbe-8bf0-b2de036449ac-kube-api-access-5xvrb\") pod \"aws-ebs-csi-driver-node-r9pms\" (UID: \"28b661ed-bfe5-4fbe-8bf0-b2de036449ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9pms"
Sep 12 00:29:25.764002 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.763978    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98rxv\" (UniqueName: \"kubernetes.io/projected/ca5a34af-39bd-486b-8293-387ff1f949e2-kube-api-access-98rxv\") pod \"multus-additional-cni-plugins-m6mrh\" (UID: \"ca5a34af-39bd-486b-8293-387ff1f949e2\") " pod="openshift-multus/multus-additional-cni-plugins-m6mrh"
Sep 12 00:29:25.764131 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.764098    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfqvk\" (UniqueName: \"kubernetes.io/projected/4d786af9-7b4e-4fd7-85bd-8e67570b009d-kube-api-access-bfqvk\") pod \"multus-mztg9\" (UID: \"4d786af9-7b4e-4fd7-85bd-8e67570b009d\") " pod="openshift-multus/multus-mztg9"
Sep 12 00:29:25.764212 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.764181    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgtc6\" (UniqueName: \"kubernetes.io/projected/c17577fa-fd1b-45a2-9d4d-6be7144ba759-kube-api-access-vgtc6\") pod \"node-ca-fd27q\" (UID: \"c17577fa-fd1b-45a2-9d4d-6be7144ba759\") " pod="openshift-image-registry/node-ca-fd27q"
Sep 12 00:29:25.764779 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.764758    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-885zl\" (UniqueName: \"kubernetes.io/projected/a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6-kube-api-access-885zl\") pod \"tuned-kltnf\" (UID: \"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6\") " pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.766034 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.766016    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbxqn\" (UniqueName: \"kubernetes.io/projected/34aa479e-166d-4d92-8909-a58f6e2649ef-kube-api-access-zbxqn\") pod \"node-resolver-gs67t\" (UID: \"34aa479e-166d-4d92-8909-a58f6e2649ef\") " pod="openshift-dns/node-resolver-gs67t"
Sep 12 00:29:25.854612 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.854542    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcqxx\" (UniqueName: \"kubernetes.io/projected/746054e5-684e-4c55-824b-6fdde562eb02-kube-api-access-zcqxx\") pod \"network-metrics-daemon-7tkvx\" (UID: \"746054e5-684e-4c55-824b-6fdde562eb02\") " pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:29:25.854612 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.854578    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ed1e49a7-a9ae-4e55-8a01-c88326238a56-dbus\") pod \"global-pull-secret-syncer-b2hjw\" (UID: \"ed1e49a7-a9ae-4e55-8a01-c88326238a56\") " pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:29:25.854612 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.854599    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ed1e49a7-a9ae-4e55-8a01-c88326238a56-original-pull-secret\") pod \"global-pull-secret-syncer-b2hjw\" (UID: \"ed1e49a7-a9ae-4e55-8a01-c88326238a56\") " pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:29:25.854864 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.854618    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/746054e5-684e-4c55-824b-6fdde562eb02-metrics-certs\") pod \"network-metrics-daemon-7tkvx\" (UID: \"746054e5-684e-4c55-824b-6fdde562eb02\") " pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:29:25.854864 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.854701    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ed1e49a7-a9ae-4e55-8a01-c88326238a56-dbus\") pod \"global-pull-secret-syncer-b2hjw\" (UID: \"ed1e49a7-a9ae-4e55-8a01-c88326238a56\") " pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:29:25.854864 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:25.854719    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:29:25.854864 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:25.854720    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 12 00:29:25.854864 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.854705    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwrlm\" (UniqueName: \"kubernetes.io/projected/550df110-4473-4487-b318-d72d9babcaa3-kube-api-access-kwrlm\") pod \"network-check-target-f6t44\" (UID: \"550df110-4473-4487-b318-d72d9babcaa3\") " pod="openshift-network-diagnostics/network-check-target-f6t44"
Sep 12 00:29:25.854864 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.854776    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ed1e49a7-a9ae-4e55-8a01-c88326238a56-kubelet-config\") pod \"global-pull-secret-syncer-b2hjw\" (UID: \"ed1e49a7-a9ae-4e55-8a01-c88326238a56\") " pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:29:25.854864 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:25.854791    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed1e49a7-a9ae-4e55-8a01-c88326238a56-original-pull-secret podName:ed1e49a7-a9ae-4e55-8a01-c88326238a56 nodeName:}" failed. No retries permitted until 2026-09-12 00:29:26.35476832 +0000 UTC m=+3.028827159 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ed1e49a7-a9ae-4e55-8a01-c88326238a56-original-pull-secret") pod "global-pull-secret-syncer-b2hjw" (UID: "ed1e49a7-a9ae-4e55-8a01-c88326238a56") : object "kube-system"/"original-pull-secret" not registered
Sep 12 00:29:25.854864 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:25.854812    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/746054e5-684e-4c55-824b-6fdde562eb02-metrics-certs podName:746054e5-684e-4c55-824b-6fdde562eb02 nodeName:}" failed. No retries permitted until 2026-09-12 00:29:26.354802886 +0000 UTC m=+3.028861731 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/746054e5-684e-4c55-824b-6fdde562eb02-metrics-certs") pod "network-metrics-daemon-7tkvx" (UID: "746054e5-684e-4c55-824b-6fdde562eb02") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:29:25.854864 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.854826    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ed1e49a7-a9ae-4e55-8a01-c88326238a56-kubelet-config\") pod \"global-pull-secret-syncer-b2hjw\" (UID: \"ed1e49a7-a9ae-4e55-8a01-c88326238a56\") " pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:29:25.860667 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:25.860641    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 12 00:29:25.860667 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:25.860670    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 12 00:29:25.860844 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:25.860687    2558 projected.go:196] Error preparing data for projected volume kube-api-access-kwrlm for pod openshift-network-diagnostics/network-check-target-f6t44: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:29:25.860844 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:25.860761    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/550df110-4473-4487-b318-d72d9babcaa3-kube-api-access-kwrlm podName:550df110-4473-4487-b318-d72d9babcaa3 nodeName:}" failed. No retries permitted until 2026-09-12 00:29:26.360739094 +0000 UTC m=+3.034797938 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kwrlm" (UniqueName: "kubernetes.io/projected/550df110-4473-4487-b318-d72d9babcaa3-kube-api-access-kwrlm") pod "network-check-target-f6t44" (UID: "550df110-4473-4487-b318-d72d9babcaa3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:29:25.863144 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.863120    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcqxx\" (UniqueName: \"kubernetes.io/projected/746054e5-684e-4c55-824b-6fdde562eb02-kube-api-access-zcqxx\") pod \"network-metrics-daemon-7tkvx\" (UID: \"746054e5-684e-4c55-824b-6fdde562eb02\") " pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:29:25.938729 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.938684    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fd27q"
Sep 12 00:29:25.945021 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.944999    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9dh42"
Sep 12 00:29:25.957249 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.957223    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-llf45"
Sep 12 00:29:25.963061 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.963038    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:25.970660 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.970638    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kltnf"
Sep 12 00:29:25.977414 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.977395    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gs67t"
Sep 12 00:29:25.984214 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.984189    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-m6mrh"
Sep 12 00:29:25.991383 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.991364    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9pms"
Sep 12 00:29:25.997222 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:25.997199    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mztg9"
Sep 12 00:29:26.033410 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:26.033377    2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 12 00:29:26.358414 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:26.358380    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ed1e49a7-a9ae-4e55-8a01-c88326238a56-original-pull-secret\") pod \"global-pull-secret-syncer-b2hjw\" (UID: \"ed1e49a7-a9ae-4e55-8a01-c88326238a56\") " pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:29:26.358608 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:26.358425    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/746054e5-684e-4c55-824b-6fdde562eb02-metrics-certs\") pod \"network-metrics-daemon-7tkvx\" (UID: \"746054e5-684e-4c55-824b-6fdde562eb02\") " pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:29:26.358608 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:26.358555    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:29:26.358608 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:26.358558    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 12 00:29:26.358782 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:26.358628    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/746054e5-684e-4c55-824b-6fdde562eb02-metrics-certs podName:746054e5-684e-4c55-824b-6fdde562eb02 nodeName:}" failed. No retries permitted until 2026-09-12 00:29:27.358606435 +0000 UTC m=+4.032665298 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/746054e5-684e-4c55-824b-6fdde562eb02-metrics-certs") pod "network-metrics-daemon-7tkvx" (UID: "746054e5-684e-4c55-824b-6fdde562eb02") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:29:26.358782 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:26.358651    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed1e49a7-a9ae-4e55-8a01-c88326238a56-original-pull-secret podName:ed1e49a7-a9ae-4e55-8a01-c88326238a56 nodeName:}" failed. No retries permitted until 2026-09-12 00:29:27.358640518 +0000 UTC m=+4.032699357 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ed1e49a7-a9ae-4e55-8a01-c88326238a56-original-pull-secret") pod "global-pull-secret-syncer-b2hjw" (UID: "ed1e49a7-a9ae-4e55-8a01-c88326238a56") : object "kube-system"/"original-pull-secret" not registered
Sep 12 00:29:26.459470 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:26.459431    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwrlm\" (UniqueName: \"kubernetes.io/projected/550df110-4473-4487-b318-d72d9babcaa3-kube-api-access-kwrlm\") pod \"network-check-target-f6t44\" (UID: \"550df110-4473-4487-b318-d72d9babcaa3\") " pod="openshift-network-diagnostics/network-check-target-f6t44"
Sep 12 00:29:26.459610 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:26.459576    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 12 00:29:26.459610 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:26.459593    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 12 00:29:26.459610 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:26.459602    2558 projected.go:196] Error preparing data for projected volume kube-api-access-kwrlm for pod openshift-network-diagnostics/network-check-target-f6t44: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:29:26.459727 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:26.459656    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/550df110-4473-4487-b318-d72d9babcaa3-kube-api-access-kwrlm podName:550df110-4473-4487-b318-d72d9babcaa3 nodeName:}" failed. No retries permitted until 2026-09-12 00:29:27.459640271 +0000 UTC m=+4.133699115 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-kwrlm" (UniqueName: "kubernetes.io/projected/550df110-4473-4487-b318-d72d9babcaa3-kube-api-access-kwrlm") pod "network-check-target-f6t44" (UID: "550df110-4473-4487-b318-d72d9babcaa3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:29:26.572650 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:26.572346    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d786af9_7b4e_4fd7_85bd_8e67570b009d.slice/crio-67dceb24581914dd3be9dffa8d1d4f84960b78cf7368a900590c32fffe851b6d WatchSource:0}: Error finding container 67dceb24581914dd3be9dffa8d1d4f84960b78cf7368a900590c32fffe851b6d: Status 404 returned error can't find the container with id 67dceb24581914dd3be9dffa8d1d4f84960b78cf7368a900590c32fffe851b6d
Sep 12 00:29:26.578586 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:26.578555    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc17577fa_fd1b_45a2_9d4d_6be7144ba759.slice/crio-18d8d6259d89d951ae19806af8f6bcebca3fe0a102bbea22f43a70ecbfa1869c WatchSource:0}: Error finding container 18d8d6259d89d951ae19806af8f6bcebca3fe0a102bbea22f43a70ecbfa1869c: Status 404 returned error can't find the container with id 18d8d6259d89d951ae19806af8f6bcebca3fe0a102bbea22f43a70ecbfa1869c
Sep 12 00:29:26.579468 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:26.579440    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28b661ed_bfe5_4fbe_8bf0_b2de036449ac.slice/crio-3aa99561eb7b70e8d5943224734d0321d1f142ac474db70b220ba11815560027 WatchSource:0}: Error finding container 3aa99561eb7b70e8d5943224734d0321d1f142ac474db70b220ba11815560027: Status 404 returned error can't find the container with id 3aa99561eb7b70e8d5943224734d0321d1f142ac474db70b220ba11815560027
Sep 12 00:29:26.580485 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:26.580051    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc3d5554_d660_453c_bc3e_4d3ed765400e.slice/crio-38ea29be4a6d4bdf7efb3e3f10acec9ad98a6b224718d8383b4c45891765cfb1 WatchSource:0}: Error finding container 38ea29be4a6d4bdf7efb3e3f10acec9ad98a6b224718d8383b4c45891765cfb1: Status 404 returned error can't find the container with id 38ea29be4a6d4bdf7efb3e3f10acec9ad98a6b224718d8383b4c45891765cfb1
Sep 12 00:29:26.584194 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:26.581875    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca5a34af_39bd_486b_8293_387ff1f949e2.slice/crio-654f1ad2ee349b3c74e545a69d723266ff312716b8e2967fcd5a7660bde6b81c WatchSource:0}: Error finding container 654f1ad2ee349b3c74e545a69d723266ff312716b8e2967fcd5a7660bde6b81c: Status 404 returned error can't find the container with id 654f1ad2ee349b3c74e545a69d723266ff312716b8e2967fcd5a7660bde6b81c
Sep 12 00:29:26.584194 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:26.582370    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45051c76_e610_4f10_bb3f_521031f82611.slice/crio-b67a59b9eb9e1968766b786d1b22dd546fcab9a1499606434cf7e58bb1f21849 WatchSource:0}: Error finding container b67a59b9eb9e1968766b786d1b22dd546fcab9a1499606434cf7e58bb1f21849: Status 404 returned error can't find the container with id b67a59b9eb9e1968766b786d1b22dd546fcab9a1499606434cf7e58bb1f21849
Sep 12 00:29:26.585022 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:26.584996    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cc8a470_64c8_4446_9e7a_994fb6309a10.slice/crio-4ba9ea3f230aaf92d7ff218ba49bc1b709996f408c5e9b88b08e23b2b4e4b1e8 WatchSource:0}: Error finding container 4ba9ea3f230aaf92d7ff218ba49bc1b709996f408c5e9b88b08e23b2b4e4b1e8: Status 404 returned error can't find the container with id 4ba9ea3f230aaf92d7ff218ba49bc1b709996f408c5e9b88b08e23b2b4e4b1e8
Sep 12 00:29:26.586025 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:26.585871    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0e36d0b_3bb7_4cb1_a68f_299d32c5bcd6.slice/crio-f0c12ce1ed3e8ec858bc336a4a2e406de7ddb056a440e70a5575b78956011f9d WatchSource:0}: Error finding container f0c12ce1ed3e8ec858bc336a4a2e406de7ddb056a440e70a5575b78956011f9d: Status 404 returned error can't find the container with id f0c12ce1ed3e8ec858bc336a4a2e406de7ddb056a440e70a5575b78956011f9d
Sep 12 00:29:26.675868 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:26.675820    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-11 00:24:25 +0000 UTC" deadline="2028-05-23 03:05:53.703202458 +0000 UTC"
Sep 12 00:29:26.675868 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:26.675864    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14858h36m27.027340791s"
Sep 12 00:29:26.741391 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:26.741355    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kltnf" event={"ID":"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6","Type":"ContainerStarted","Data":"f0c12ce1ed3e8ec858bc336a4a2e406de7ddb056a440e70a5575b78956011f9d"}
Sep 12 00:29:26.742680 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:26.742625    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9pms" event={"ID":"28b661ed-bfe5-4fbe-8bf0-b2de036449ac","Type":"ContainerStarted","Data":"3aa99561eb7b70e8d5943224734d0321d1f142ac474db70b220ba11815560027"}
Sep 12 00:29:26.743888 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:26.743862    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gs67t" event={"ID":"34aa479e-166d-4d92-8909-a58f6e2649ef","Type":"ContainerStarted","Data":"c0d4da39918d4b5c27d2dfc4ab846823ef1658d7e89c6789ae49c0144df8b314"}
Sep 12 00:29:26.745062 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:26.745042    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mztg9" event={"ID":"4d786af9-7b4e-4fd7-85bd-8e67570b009d","Type":"ContainerStarted","Data":"67dceb24581914dd3be9dffa8d1d4f84960b78cf7368a900590c32fffe851b6d"}
Sep 12 00:29:26.747030 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:26.747006    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-85.ec2.internal" event={"ID":"83844964b8d09ab82b7ac30298c1090d","Type":"ContainerStarted","Data":"01bdec0ed066d47f855d9a239773d43a10c0a58f85e4bf1c50a99f224834033f"}
Sep 12 00:29:26.748410 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:26.748384    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-llf45" event={"ID":"7cc8a470-64c8-4446-9e7a-994fb6309a10","Type":"ContainerStarted","Data":"4ba9ea3f230aaf92d7ff218ba49bc1b709996f408c5e9b88b08e23b2b4e4b1e8"}
Sep 12 00:29:26.749525 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:26.749505    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p" event={"ID":"45051c76-e610-4f10-bb3f-521031f82611","Type":"ContainerStarted","Data":"b67a59b9eb9e1968766b786d1b22dd546fcab9a1499606434cf7e58bb1f21849"}
Sep 12 00:29:26.750811 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:26.750784    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m6mrh" event={"ID":"ca5a34af-39bd-486b-8293-387ff1f949e2","Type":"ContainerStarted","Data":"654f1ad2ee349b3c74e545a69d723266ff312716b8e2967fcd5a7660bde6b81c"}
Sep 12 00:29:26.752019 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:26.751998    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9dh42" event={"ID":"cc3d5554-d660-453c-bc3e-4d3ed765400e","Type":"ContainerStarted","Data":"38ea29be4a6d4bdf7efb3e3f10acec9ad98a6b224718d8383b4c45891765cfb1"}
Sep 12 00:29:26.753140 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:26.753112    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fd27q" event={"ID":"c17577fa-fd1b-45a2-9d4d-6be7144ba759","Type":"ContainerStarted","Data":"18d8d6259d89d951ae19806af8f6bcebca3fe0a102bbea22f43a70ecbfa1869c"}
Sep 12 00:29:26.764803 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:26.764763    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-85.ec2.internal" podStartSLOduration=1.764750936 podStartE2EDuration="1.764750936s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:29:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:29:26.764412391 +0000 UTC m=+3.438471253" watchObservedRunningTime="2026-09-12 00:29:26.764750936 +0000 UTC m=+3.438809794"
Sep 12 00:29:27.366580 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:27.366541    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ed1e49a7-a9ae-4e55-8a01-c88326238a56-original-pull-secret\") pod \"global-pull-secret-syncer-b2hjw\" (UID: \"ed1e49a7-a9ae-4e55-8a01-c88326238a56\") " pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:29:27.366779 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:27.366606    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/746054e5-684e-4c55-824b-6fdde562eb02-metrics-certs\") pod \"network-metrics-daemon-7tkvx\" (UID: \"746054e5-684e-4c55-824b-6fdde562eb02\") " pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:29:27.366847 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:27.366793    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:29:27.366901 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:27.366865    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/746054e5-684e-4c55-824b-6fdde562eb02-metrics-certs podName:746054e5-684e-4c55-824b-6fdde562eb02 nodeName:}" failed. No retries permitted until 2026-09-12 00:29:29.366844806 +0000 UTC m=+6.040903651 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/746054e5-684e-4c55-824b-6fdde562eb02-metrics-certs") pod "network-metrics-daemon-7tkvx" (UID: "746054e5-684e-4c55-824b-6fdde562eb02") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:29:27.367342 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:27.367322    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 12 00:29:27.367437 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:27.367383    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed1e49a7-a9ae-4e55-8a01-c88326238a56-original-pull-secret podName:ed1e49a7-a9ae-4e55-8a01-c88326238a56 nodeName:}" failed. No retries permitted until 2026-09-12 00:29:29.367366516 +0000 UTC m=+6.041425358 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ed1e49a7-a9ae-4e55-8a01-c88326238a56-original-pull-secret") pod "global-pull-secret-syncer-b2hjw" (UID: "ed1e49a7-a9ae-4e55-8a01-c88326238a56") : object "kube-system"/"original-pull-secret" not registered
Sep 12 00:29:27.468015 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:27.467975    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwrlm\" (UniqueName: \"kubernetes.io/projected/550df110-4473-4487-b318-d72d9babcaa3-kube-api-access-kwrlm\") pod \"network-check-target-f6t44\" (UID: \"550df110-4473-4487-b318-d72d9babcaa3\") " pod="openshift-network-diagnostics/network-check-target-f6t44"
Sep 12 00:29:27.468235 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:27.468208    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 12 00:29:27.468235 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:27.468231    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 12 00:29:27.468359 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:27.468245    2558 projected.go:196] Error preparing data for projected volume kube-api-access-kwrlm for pod openshift-network-diagnostics/network-check-target-f6t44: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:29:27.468359 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:27.468311    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/550df110-4473-4487-b318-d72d9babcaa3-kube-api-access-kwrlm podName:550df110-4473-4487-b318-d72d9babcaa3 nodeName:}" failed. No retries permitted until 2026-09-12 00:29:29.468291236 +0000 UTC m=+6.142350077 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-kwrlm" (UniqueName: "kubernetes.io/projected/550df110-4473-4487-b318-d72d9babcaa3-kube-api-access-kwrlm") pod "network-check-target-f6t44" (UID: "550df110-4473-4487-b318-d72d9babcaa3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:29:27.734328 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:27.734247    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:29:27.734936 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:27.734406    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tkvx" podUID="746054e5-684e-4c55-824b-6fdde562eb02"
Sep 12 00:29:27.734936 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:27.734869    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f6t44"
Sep 12 00:29:27.735059 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:27.734962    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f6t44" podUID="550df110-4473-4487-b318-d72d9babcaa3"
Sep 12 00:29:27.735059 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:27.735023    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:29:27.735246 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:27.735097    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2hjw" podUID="ed1e49a7-a9ae-4e55-8a01-c88326238a56"
Sep 12 00:29:27.761820 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:27.761775    2558 generic.go:356] "Generic (PLEG): container finished" podID="6b08168807839ba0280c207cb77632af" containerID="ab2b3ab2aa568704f6532f191a0ffb3b2fe1846add4160fed6623c11e7de5966" exitCode=0
Sep 12 00:29:27.762421 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:27.762381    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-85.ec2.internal" event={"ID":"6b08168807839ba0280c207cb77632af","Type":"ContainerDied","Data":"ab2b3ab2aa568704f6532f191a0ffb3b2fe1846add4160fed6623c11e7de5966"}
Sep 12 00:29:28.802910 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:28.802841    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-85.ec2.internal" event={"ID":"6b08168807839ba0280c207cb77632af","Type":"ContainerStarted","Data":"9b0868e6f5aa6f58a7558d385f3f9fd96523868234780a04b71feb621733b0e1"}
Sep 12 00:29:29.383479 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:29.383419    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ed1e49a7-a9ae-4e55-8a01-c88326238a56-original-pull-secret\") pod \"global-pull-secret-syncer-b2hjw\" (UID: \"ed1e49a7-a9ae-4e55-8a01-c88326238a56\") " pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:29:29.383479 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:29.383481    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/746054e5-684e-4c55-824b-6fdde562eb02-metrics-certs\") pod \"network-metrics-daemon-7tkvx\" (UID: \"746054e5-684e-4c55-824b-6fdde562eb02\") " pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:29:29.383726 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:29.383641    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:29:29.383726 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:29.383712    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/746054e5-684e-4c55-824b-6fdde562eb02-metrics-certs podName:746054e5-684e-4c55-824b-6fdde562eb02 nodeName:}" failed. No retries permitted until 2026-09-12 00:29:33.383690027 +0000 UTC m=+10.057748889 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/746054e5-684e-4c55-824b-6fdde562eb02-metrics-certs") pod "network-metrics-daemon-7tkvx" (UID: "746054e5-684e-4c55-824b-6fdde562eb02") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:29:29.384237 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:29.384214    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 12 00:29:29.384341 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:29.384277    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed1e49a7-a9ae-4e55-8a01-c88326238a56-original-pull-secret podName:ed1e49a7-a9ae-4e55-8a01-c88326238a56 nodeName:}" failed. No retries permitted until 2026-09-12 00:29:33.384260662 +0000 UTC m=+10.058319506 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ed1e49a7-a9ae-4e55-8a01-c88326238a56-original-pull-secret") pod "global-pull-secret-syncer-b2hjw" (UID: "ed1e49a7-a9ae-4e55-8a01-c88326238a56") : object "kube-system"/"original-pull-secret" not registered
Sep 12 00:29:29.483968 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:29.483928    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwrlm\" (UniqueName: \"kubernetes.io/projected/550df110-4473-4487-b318-d72d9babcaa3-kube-api-access-kwrlm\") pod \"network-check-target-f6t44\" (UID: \"550df110-4473-4487-b318-d72d9babcaa3\") " pod="openshift-network-diagnostics/network-check-target-f6t44"
Sep 12 00:29:29.484261 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:29.484142    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 12 00:29:29.484261 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:29.484168    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 12 00:29:29.484261 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:29.484182    2558 projected.go:196] Error preparing data for projected volume kube-api-access-kwrlm for pod openshift-network-diagnostics/network-check-target-f6t44: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:29:29.484261 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:29.484244    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/550df110-4473-4487-b318-d72d9babcaa3-kube-api-access-kwrlm podName:550df110-4473-4487-b318-d72d9babcaa3 nodeName:}" failed. No retries permitted until 2026-09-12 00:29:33.484223619 +0000 UTC m=+10.158282475 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-kwrlm" (UniqueName: "kubernetes.io/projected/550df110-4473-4487-b318-d72d9babcaa3-kube-api-access-kwrlm") pod "network-check-target-f6t44" (UID: "550df110-4473-4487-b318-d72d9babcaa3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:29:29.734105 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:29.733302    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:29:29.734105 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:29.733458    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tkvx" podUID="746054e5-684e-4c55-824b-6fdde562eb02"
Sep 12 00:29:29.734105 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:29.733866    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f6t44"
Sep 12 00:29:29.734105 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:29.733956    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f6t44" podUID="550df110-4473-4487-b318-d72d9babcaa3"
Sep 12 00:29:29.734765 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:29.734584    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:29:29.734765 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:29.734685    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2hjw" podUID="ed1e49a7-a9ae-4e55-8a01-c88326238a56"
Sep 12 00:29:31.733246 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:31.733212    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:29:31.733737 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:31.733253    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:29:31.733737 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:31.733213    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f6t44"
Sep 12 00:29:31.733737 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:31.733368    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tkvx" podUID="746054e5-684e-4c55-824b-6fdde562eb02"
Sep 12 00:29:31.733737 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:31.733450    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f6t44" podUID="550df110-4473-4487-b318-d72d9babcaa3"
Sep 12 00:29:31.733737 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:31.733534    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2hjw" podUID="ed1e49a7-a9ae-4e55-8a01-c88326238a56"
Sep 12 00:29:33.416960 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:33.416910    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ed1e49a7-a9ae-4e55-8a01-c88326238a56-original-pull-secret\") pod \"global-pull-secret-syncer-b2hjw\" (UID: \"ed1e49a7-a9ae-4e55-8a01-c88326238a56\") " pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:29:33.417417 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:33.416975    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/746054e5-684e-4c55-824b-6fdde562eb02-metrics-certs\") pod \"network-metrics-daemon-7tkvx\" (UID: \"746054e5-684e-4c55-824b-6fdde562eb02\") " pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:29:33.417417 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:33.417103    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:29:33.417417 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:33.417203    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/746054e5-684e-4c55-824b-6fdde562eb02-metrics-certs podName:746054e5-684e-4c55-824b-6fdde562eb02 nodeName:}" failed. No retries permitted until 2026-09-12 00:29:41.417180788 +0000 UTC m=+18.091239629 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/746054e5-684e-4c55-824b-6fdde562eb02-metrics-certs") pod "network-metrics-daemon-7tkvx" (UID: "746054e5-684e-4c55-824b-6fdde562eb02") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:29:33.417417 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:33.417261    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 12 00:29:33.417417 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:33.417339    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed1e49a7-a9ae-4e55-8a01-c88326238a56-original-pull-secret podName:ed1e49a7-a9ae-4e55-8a01-c88326238a56 nodeName:}" failed. No retries permitted until 2026-09-12 00:29:41.417319473 +0000 UTC m=+18.091378316 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ed1e49a7-a9ae-4e55-8a01-c88326238a56-original-pull-secret") pod "global-pull-secret-syncer-b2hjw" (UID: "ed1e49a7-a9ae-4e55-8a01-c88326238a56") : object "kube-system"/"original-pull-secret" not registered
Sep 12 00:29:33.519006 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:33.518357    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwrlm\" (UniqueName: \"kubernetes.io/projected/550df110-4473-4487-b318-d72d9babcaa3-kube-api-access-kwrlm\") pod \"network-check-target-f6t44\" (UID: \"550df110-4473-4487-b318-d72d9babcaa3\") " pod="openshift-network-diagnostics/network-check-target-f6t44"
Sep 12 00:29:33.519006 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:33.518532    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 12 00:29:33.519006 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:33.518556    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 12 00:29:33.519006 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:33.518569    2558 projected.go:196] Error preparing data for projected volume kube-api-access-kwrlm for pod openshift-network-diagnostics/network-check-target-f6t44: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:29:33.519006 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:33.518632    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/550df110-4473-4487-b318-d72d9babcaa3-kube-api-access-kwrlm podName:550df110-4473-4487-b318-d72d9babcaa3 nodeName:}" failed. No retries permitted until 2026-09-12 00:29:41.518613293 +0000 UTC m=+18.192672136 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-kwrlm" (UniqueName: "kubernetes.io/projected/550df110-4473-4487-b318-d72d9babcaa3-kube-api-access-kwrlm") pod "network-check-target-f6t44" (UID: "550df110-4473-4487-b318-d72d9babcaa3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:29:33.732969 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:33.732887    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:29:33.733158 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:33.733031    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tkvx" podUID="746054e5-684e-4c55-824b-6fdde562eb02"
Sep 12 00:29:33.733764 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:33.733470    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f6t44"
Sep 12 00:29:33.733764 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:33.733570    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f6t44" podUID="550df110-4473-4487-b318-d72d9babcaa3"
Sep 12 00:29:33.733764 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:33.733635    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:29:33.733764 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:33.733710    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2hjw" podUID="ed1e49a7-a9ae-4e55-8a01-c88326238a56"
Sep 12 00:29:35.732132 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:35.732063    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f6t44"
Sep 12 00:29:35.732630 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:35.732063    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:29:35.732630 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:35.732211    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f6t44" podUID="550df110-4473-4487-b318-d72d9babcaa3"
Sep 12 00:29:35.732630 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:35.732319    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tkvx" podUID="746054e5-684e-4c55-824b-6fdde562eb02"
Sep 12 00:29:35.732630 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:35.732064    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:29:35.732630 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:35.732429    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2hjw" podUID="ed1e49a7-a9ae-4e55-8a01-c88326238a56"
Sep 12 00:29:37.732033 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:37.731995    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:29:37.732430 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:37.731995    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:29:37.732430 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:37.732129    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f6t44"
Sep 12 00:29:37.732430 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:37.732131    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tkvx" podUID="746054e5-684e-4c55-824b-6fdde562eb02"
Sep 12 00:29:37.732430 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:37.732200    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2hjw" podUID="ed1e49a7-a9ae-4e55-8a01-c88326238a56"
Sep 12 00:29:37.732430 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:37.732251    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f6t44" podUID="550df110-4473-4487-b318-d72d9babcaa3"
Sep 12 00:29:39.732427 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:39.732392    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f6t44"
Sep 12 00:29:39.732427 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:39.732423    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:29:39.732921 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:39.732392    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:29:39.732921 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:39.732524    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f6t44" podUID="550df110-4473-4487-b318-d72d9babcaa3"
Sep 12 00:29:39.732921 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:39.732631    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tkvx" podUID="746054e5-684e-4c55-824b-6fdde562eb02"
Sep 12 00:29:39.732921 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:39.732720    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2hjw" podUID="ed1e49a7-a9ae-4e55-8a01-c88326238a56"
Sep 12 00:29:41.476324 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:41.476103    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ed1e49a7-a9ae-4e55-8a01-c88326238a56-original-pull-secret\") pod \"global-pull-secret-syncer-b2hjw\" (UID: \"ed1e49a7-a9ae-4e55-8a01-c88326238a56\") " pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:29:41.476861 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:41.476355    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/746054e5-684e-4c55-824b-6fdde562eb02-metrics-certs\") pod \"network-metrics-daemon-7tkvx\" (UID: \"746054e5-684e-4c55-824b-6fdde562eb02\") " pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:29:41.476861 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:41.476219    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 12 00:29:41.476861 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:41.476443    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:29:41.476861 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:41.476493    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed1e49a7-a9ae-4e55-8a01-c88326238a56-original-pull-secret podName:ed1e49a7-a9ae-4e55-8a01-c88326238a56 nodeName:}" failed. No retries permitted until 2026-09-12 00:29:57.476466218 +0000 UTC m=+34.150525059 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ed1e49a7-a9ae-4e55-8a01-c88326238a56-original-pull-secret") pod "global-pull-secret-syncer-b2hjw" (UID: "ed1e49a7-a9ae-4e55-8a01-c88326238a56") : object "kube-system"/"original-pull-secret" not registered
Sep 12 00:29:41.476861 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:41.476514    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/746054e5-684e-4c55-824b-6fdde562eb02-metrics-certs podName:746054e5-684e-4c55-824b-6fdde562eb02 nodeName:}" failed. No retries permitted until 2026-09-12 00:29:57.476503457 +0000 UTC m=+34.150562298 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/746054e5-684e-4c55-824b-6fdde562eb02-metrics-certs") pod "network-metrics-daemon-7tkvx" (UID: "746054e5-684e-4c55-824b-6fdde562eb02") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:29:41.577676 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:41.577642    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwrlm\" (UniqueName: \"kubernetes.io/projected/550df110-4473-4487-b318-d72d9babcaa3-kube-api-access-kwrlm\") pod \"network-check-target-f6t44\" (UID: \"550df110-4473-4487-b318-d72d9babcaa3\") " pod="openshift-network-diagnostics/network-check-target-f6t44"
Sep 12 00:29:41.577850 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:41.577799    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 12 00:29:41.577850 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:41.577815    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 12 00:29:41.577850 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:41.577827    2558 projected.go:196] Error preparing data for projected volume kube-api-access-kwrlm for pod openshift-network-diagnostics/network-check-target-f6t44: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:29:41.577983 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:41.577885    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/550df110-4473-4487-b318-d72d9babcaa3-kube-api-access-kwrlm podName:550df110-4473-4487-b318-d72d9babcaa3 nodeName:}" failed. No retries permitted until 2026-09-12 00:29:57.577867912 +0000 UTC m=+34.251926767 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-kwrlm" (UniqueName: "kubernetes.io/projected/550df110-4473-4487-b318-d72d9babcaa3-kube-api-access-kwrlm") pod "network-check-target-f6t44" (UID: "550df110-4473-4487-b318-d72d9babcaa3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:29:41.732140 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:41.732040    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:29:41.732296 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:41.732040    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f6t44"
Sep 12 00:29:41.732296 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:41.732176    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tkvx" podUID="746054e5-684e-4c55-824b-6fdde562eb02"
Sep 12 00:29:41.732296 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:41.732184    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:29:41.732296 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:41.732263    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f6t44" podUID="550df110-4473-4487-b318-d72d9babcaa3"
Sep 12 00:29:41.732528 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:41.732323    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2hjw" podUID="ed1e49a7-a9ae-4e55-8a01-c88326238a56"
Sep 12 00:29:43.732501 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:43.732471    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:29:43.732956 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:43.732585    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:29:43.732956 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:43.732711    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tkvx" podUID="746054e5-684e-4c55-824b-6fdde562eb02"
Sep 12 00:29:43.732956 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:43.732583    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2hjw" podUID="ed1e49a7-a9ae-4e55-8a01-c88326238a56"
Sep 12 00:29:43.732956 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:43.732760    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f6t44"
Sep 12 00:29:43.732956 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:43.732807    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f6t44" podUID="550df110-4473-4487-b318-d72d9babcaa3"
Sep 12 00:29:45.731622 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:45.731362    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f6t44"
Sep 12 00:29:45.732126 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:45.731418    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:29:45.732126 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:45.731728    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f6t44" podUID="550df110-4473-4487-b318-d72d9babcaa3"
Sep 12 00:29:45.732126 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:45.731452    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:29:45.732126 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:45.731810    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tkvx" podUID="746054e5-684e-4c55-824b-6fdde562eb02"
Sep 12 00:29:45.732126 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:45.731867    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2hjw" podUID="ed1e49a7-a9ae-4e55-8a01-c88326238a56"
Sep 12 00:29:45.843709 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:45.843673    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p" event={"ID":"45051c76-e610-4f10-bb3f-521031f82611","Type":"ContainerStarted","Data":"fa09bba27af6f70f130f9fdf366dd2d6c0c9a0f6ba13e4f1c227153c0bb3eb52"}
Sep 12 00:29:45.843896 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:45.843754    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p" event={"ID":"45051c76-e610-4f10-bb3f-521031f82611","Type":"ContainerStarted","Data":"9ded0fe1a72b33117a77756162389792668369e55e7c5e43ff48d152974792a1"}
Sep 12 00:29:45.843896 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:45.843804    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p" event={"ID":"45051c76-e610-4f10-bb3f-521031f82611","Type":"ContainerStarted","Data":"a025c5da171645d968417937585c54f4b5bc33d0598271c88bf62c4a7837452b"}
Sep 12 00:29:45.843896 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:45.843835    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p" event={"ID":"45051c76-e610-4f10-bb3f-521031f82611","Type":"ContainerStarted","Data":"422ac880866fd6ccc1c8583b64174e62fc6fd355cbf82426a14566434c630d03"}
Sep 12 00:29:45.843896 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:45.843865    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p" event={"ID":"45051c76-e610-4f10-bb3f-521031f82611","Type":"ContainerStarted","Data":"e6922ce64bf4e4818d76ffe75dec101c401adb5ea9a3da7e396d7fec402d3db0"}
Sep 12 00:29:45.844057 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:45.843896    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p" event={"ID":"45051c76-e610-4f10-bb3f-521031f82611","Type":"ContainerStarted","Data":"c89e54496346800c7efbe9f31f198f0671a52dd8ce4aee54ef0c72dd73820643"}
Sep 12 00:29:45.845302 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:45.845271    2558 generic.go:356] "Generic (PLEG): container finished" podID="ca5a34af-39bd-486b-8293-387ff1f949e2" containerID="e3f73904a1f5804b6043a4af6b187b491ffdf6753b8afb5129cb518121c33565" exitCode=0
Sep 12 00:29:45.845420 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:45.845347    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m6mrh" event={"ID":"ca5a34af-39bd-486b-8293-387ff1f949e2","Type":"ContainerDied","Data":"e3f73904a1f5804b6043a4af6b187b491ffdf6753b8afb5129cb518121c33565"}
Sep 12 00:29:45.846876 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:45.846840    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9dh42" event={"ID":"cc3d5554-d660-453c-bc3e-4d3ed765400e","Type":"ContainerStarted","Data":"8cada6d657b3c88ba4951e1b7e708df99d510b3bb31178c35c17ce95b766623a"}
Sep 12 00:29:45.848517 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:45.848493    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fd27q" event={"ID":"c17577fa-fd1b-45a2-9d4d-6be7144ba759","Type":"ContainerStarted","Data":"d307fe5ff80d73b65dfd0939d14f253db09430759dfa514bf727b78cb80e4b18"}
Sep 12 00:29:45.849887 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:45.849855    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kltnf" event={"ID":"a0e36d0b-3bb7-4cb1-a68f-299d32c5bcd6","Type":"ContainerStarted","Data":"5c11168dc279bb055df1126db4a4c9cd56e80b06ab95a3ea32a0c0788e4ddb4c"}
Sep 12 00:29:45.851279 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:45.851261    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9pms" event={"ID":"28b661ed-bfe5-4fbe-8bf0-b2de036449ac","Type":"ContainerStarted","Data":"dc34641beca7382fded6f9ed36150dfdaea0b04304c2ffa9c7f104b496cd94b6"}
Sep 12 00:29:45.852648 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:45.852621    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gs67t" event={"ID":"34aa479e-166d-4d92-8909-a58f6e2649ef","Type":"ContainerStarted","Data":"b357660f880f6d532bf27d108332f31f61db051bd051e5a8f60debe03c0436ed"}
Sep 12 00:29:45.854196 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:45.854176    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mztg9" event={"ID":"4d786af9-7b4e-4fd7-85bd-8e67570b009d","Type":"ContainerStarted","Data":"7d177742418cbb0f418d3724848f06f4e18a17cace5565a182eb807dd4dffbc6"}
Sep 12 00:29:45.867404 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:45.867322    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-85.ec2.internal" podStartSLOduration=20.867311223 podStartE2EDuration="20.867311223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:29:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:29:28.821250913 +0000 UTC m=+5.495309790" watchObservedRunningTime="2026-09-12 00:29:45.867311223 +0000 UTC m=+22.541370081"
Sep 12 00:29:45.880907 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:45.880868    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-fd27q" podStartSLOduration=3.334720265 podStartE2EDuration="21.88085498s" totalImagesPullingTime="18.546134715s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:29:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:29:45.880833587 +0000 UTC m=+22.554892447" watchObservedRunningTime="2026-09-12 00:29:45.88085498 +0000 UTC m=+22.554913839"
Sep 12 00:29:45.894168 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:45.894122    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-gs67t" podStartSLOduration=3.348042898 podStartE2EDuration="21.894109692s" totalImagesPullingTime="18.546066794s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:29:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:29:45.893647372 +0000 UTC m=+22.567706225" watchObservedRunningTime="2026-09-12 00:29:45.894109692 +0000 UTC m=+22.568168548"
Sep 12 00:29:45.926546 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:45.926490    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-kltnf" podStartSLOduration=3.374496682 podStartE2EDuration="21.926469634s" totalImagesPullingTime="18.551972952s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:29:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:29:45.926289176 +0000 UTC m=+22.600348048" watchObservedRunningTime="2026-09-12 00:29:45.926469634 +0000 UTC m=+22.600528496"
Sep 12 00:29:45.927033 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:45.927003    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9dh42" podStartSLOduration=6.739756859 podStartE2EDuration="21.926991279s" totalImagesPullingTime="15.18723442s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:29:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:29:45.908489594 +0000 UTC m=+22.582548454" watchObservedRunningTime="2026-09-12 00:29:45.926991279 +0000 UTC m=+22.601050139"
Sep 12 00:29:45.942059 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:45.942004    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-mztg9" podStartSLOduration=3.326105576 podStartE2EDuration="21.941989466s" totalImagesPullingTime="18.61588389s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:29:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:29:45.941492382 +0000 UTC m=+22.615551238" watchObservedRunningTime="2026-09-12 00:29:45.941989466 +0000 UTC m=+22.616048326"
Sep 12 00:29:45.969319 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:45.969278    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:29:46.063930 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:46.063900    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:29:46.120888 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:46.120823    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:29:46.802030 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:46.802003    2558 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 12 00:29:46.857958 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:46.857926    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-llf45" event={"ID":"7cc8a470-64c8-4446-9e7a-994fb6309a10","Type":"ContainerStarted","Data":"17a434cb98e41bf3e59a24450fe69ce696f2623d5de377daf7f1c91357d87111"}
Sep 12 00:29:46.859879 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:46.859849    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9pms" event={"ID":"28b661ed-bfe5-4fbe-8bf0-b2de036449ac","Type":"ContainerStarted","Data":"5d961b84df6fea01aa8e593145e42111037ff08cf693847d0c3a374a703ce75c"}
Sep 12 00:29:46.874313 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:46.872496    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-llf45" podStartSLOduration=4.330905793 podStartE2EDuration="22.872477157s" totalImagesPullingTime="18.541571364s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:29:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:29:46.871363193 +0000 UTC m=+23.545422051" watchObservedRunningTime="2026-09-12 00:29:46.872477157 +0000 UTC m=+23.546536016"
Sep 12 00:29:46.988379 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:46.988349    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:29:47.038521 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:47.038441    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:29:47.165703 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:47.165678    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:29:47.222547 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:47.222496    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:29:47.715998 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:47.715841    2558 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-12T00:29:46.802024367Z","UUID":"9c1f7d70-156d-4658-9cc7-20f7fa26ef4c","Handler":null,"Name":"","Endpoint":""}
Sep 12 00:29:47.717663 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:47.717637    2558 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 12 00:29:47.717779 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:47.717678    2558 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 12 00:29:47.732426 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:47.732403    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:29:47.732559 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:47.732411    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f6t44"
Sep 12 00:29:47.732559 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:47.732519    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:29:47.732559 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:47.732515    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tkvx" podUID="746054e5-684e-4c55-824b-6fdde562eb02"
Sep 12 00:29:47.732696 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:47.732674    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f6t44" podUID="550df110-4473-4487-b318-d72d9babcaa3"
Sep 12 00:29:47.732776 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:47.732759    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2hjw" podUID="ed1e49a7-a9ae-4e55-8a01-c88326238a56"
Sep 12 00:29:48.615115 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:48.614905    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:29:48.715423 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:48.715389    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:29:48.749709 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:48.749675    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9dh42"
Sep 12 00:29:48.857891 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:48.857859    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9dh42"
Sep 12 00:29:48.858382 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:48.858367    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9dh42"
Sep 12 00:29:48.868891 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:48.868822    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p" event={"ID":"45051c76-e610-4f10-bb3f-521031f82611","Type":"ContainerStarted","Data":"27824f5562b367a8fb7310f6d8278add95549c8f5164085f4c75317ca9980289"}
Sep 12 00:29:48.871346 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:48.871323    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9pms" event={"ID":"28b661ed-bfe5-4fbe-8bf0-b2de036449ac","Type":"ContainerStarted","Data":"1c4362083c4e99f78fe763c7bc081bbb2afe4ab61526dd2ec36d7f0c20a8c1c6"}
Sep 12 00:29:48.872072 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:48.872054    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9dh42"
Sep 12 00:29:48.901015 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:48.900970    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r9pms" podStartSLOduration=4.015148333 podStartE2EDuration="24.900956065s" totalImagesPullingTime="20.885807732s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:29:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:29:48.900862385 +0000 UTC m=+25.574921247" watchObservedRunningTime="2026-09-12 00:29:48.900956065 +0000 UTC m=+25.575014920"
Sep 12 00:29:48.929834 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:48.929800    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:29:49.665376 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:49.665337    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:29:49.731940 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:49.731910    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:29:49.732140 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:49.732023    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2hjw" podUID="ed1e49a7-a9ae-4e55-8a01-c88326238a56"
Sep 12 00:29:49.732140 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:49.731910    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f6t44"
Sep 12 00:29:49.732140 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:49.732058    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:29:49.732301 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:49.732221    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tkvx" podUID="746054e5-684e-4c55-824b-6fdde562eb02"
Sep 12 00:29:49.732301 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:49.732281    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f6t44" podUID="550df110-4473-4487-b318-d72d9babcaa3"
Sep 12 00:29:49.766032 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:49.765990    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:29:50.564771 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:50.564738    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:29:51.731770 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:51.731546    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:29:51.732158 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:51.731555    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f6t44"
Sep 12 00:29:51.732158 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:51.731809    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tkvx" podUID="746054e5-684e-4c55-824b-6fdde562eb02"
Sep 12 00:29:51.732158 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:51.731894    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f6t44" podUID="550df110-4473-4487-b318-d72d9babcaa3"
Sep 12 00:29:51.732158 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:51.731557    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:29:51.732158 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:51.731998    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2hjw" podUID="ed1e49a7-a9ae-4e55-8a01-c88326238a56"
Sep 12 00:29:51.881814 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:51.881782    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p" event={"ID":"45051c76-e610-4f10-bb3f-521031f82611","Type":"ContainerStarted","Data":"5a23b4af04d08ec085f18efbf3a20281530427f5474b53bd80dd5572dd0e3ee1"}
Sep 12 00:29:51.882098 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:51.882035    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:51.882231 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:51.882131    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:51.883768 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:51.883739    2558 generic.go:356] "Generic (PLEG): container finished" podID="ca5a34af-39bd-486b-8293-387ff1f949e2" containerID="139a8bd4556cd1a2d929a1aa716ced711e7cedfa4850a01de8cf55598c547c07" exitCode=0
Sep 12 00:29:51.883867 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:51.883788    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m6mrh" event={"ID":"ca5a34af-39bd-486b-8293-387ff1f949e2","Type":"ContainerDied","Data":"139a8bd4556cd1a2d929a1aa716ced711e7cedfa4850a01de8cf55598c547c07"}
Sep 12 00:29:51.898345 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:51.898318    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:51.909057 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:51.909021    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p" podStartSLOduration=9.312379237 podStartE2EDuration="27.909009396s" totalImagesPullingTime="18.596630159s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:29:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:29:51.908891993 +0000 UTC m=+28.582950962" watchObservedRunningTime="2026-09-12 00:29:51.909009396 +0000 UTC m=+28.583068254"
Sep 12 00:29:52.886566 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:52.886486    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:52.901425 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:52.901404    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:29:53.210342 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:53.210255    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-b2hjw"]
Sep 12 00:29:53.210503 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:53.210399    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:29:53.210566 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:53.210511    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2hjw" podUID="ed1e49a7-a9ae-4e55-8a01-c88326238a56"
Sep 12 00:29:53.213397 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:53.213371    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-f6t44"]
Sep 12 00:29:53.213528 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:53.213473    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f6t44"
Sep 12 00:29:53.213592 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:53.213575    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f6t44" podUID="550df110-4473-4487-b318-d72d9babcaa3"
Sep 12 00:29:53.213976 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:53.213955    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7tkvx"]
Sep 12 00:29:53.214065 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:53.214052    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:29:53.214164 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:53.214149    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tkvx" podUID="746054e5-684e-4c55-824b-6fdde562eb02"
Sep 12 00:29:53.890101 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:53.890049    2558 generic.go:356] "Generic (PLEG): container finished" podID="ca5a34af-39bd-486b-8293-387ff1f949e2" containerID="82bd9337f26b302d44ba624e1e10cd2fa08923bedc959a53205232a99610b20b" exitCode=0
Sep 12 00:29:53.890787 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:53.890129    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m6mrh" event={"ID":"ca5a34af-39bd-486b-8293-387ff1f949e2","Type":"ContainerDied","Data":"82bd9337f26b302d44ba624e1e10cd2fa08923bedc959a53205232a99610b20b"}
Sep 12 00:29:54.732351 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:54.732166    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:29:54.732504 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:54.732170    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:29:54.732504 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:54.732440    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2hjw" podUID="ed1e49a7-a9ae-4e55-8a01-c88326238a56"
Sep 12 00:29:54.732504 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:54.732173    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f6t44"
Sep 12 00:29:54.732622 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:54.732509    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tkvx" podUID="746054e5-684e-4c55-824b-6fdde562eb02"
Sep 12 00:29:54.732622 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:54.732581    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f6t44" podUID="550df110-4473-4487-b318-d72d9babcaa3"
Sep 12 00:29:55.198663 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:55.198635    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:29:55.299030 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:55.298999    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:29:55.897074 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:55.897043    2558 generic.go:356] "Generic (PLEG): container finished" podID="ca5a34af-39bd-486b-8293-387ff1f949e2" containerID="dc10afd037e7f4e4688de6bf233c9934ade3b0491c91fdf55ceba27bcd7f4821" exitCode=0
Sep 12 00:29:55.897281 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:55.897106    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m6mrh" event={"ID":"ca5a34af-39bd-486b-8293-387ff1f949e2","Type":"ContainerDied","Data":"dc10afd037e7f4e4688de6bf233c9934ade3b0491c91fdf55ceba27bcd7f4821"}
Sep 12 00:29:56.731799 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:56.731766    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:29:56.732318 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:56.731774    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:29:56.732318 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:56.731911    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f6t44"
Sep 12 00:29:56.732318 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:56.731900    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tkvx" podUID="746054e5-684e-4c55-824b-6fdde562eb02"
Sep 12 00:29:56.732318 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:56.732022    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2hjw" podUID="ed1e49a7-a9ae-4e55-8a01-c88326238a56"
Sep 12 00:29:56.732318 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:56.732125    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f6t44" podUID="550df110-4473-4487-b318-d72d9babcaa3"
Sep 12 00:29:57.492874 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:57.492834    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ed1e49a7-a9ae-4e55-8a01-c88326238a56-original-pull-secret\") pod \"global-pull-secret-syncer-b2hjw\" (UID: \"ed1e49a7-a9ae-4e55-8a01-c88326238a56\") " pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:29:57.493109 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:57.492885    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/746054e5-684e-4c55-824b-6fdde562eb02-metrics-certs\") pod \"network-metrics-daemon-7tkvx\" (UID: \"746054e5-684e-4c55-824b-6fdde562eb02\") " pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:29:57.493109 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:57.493001    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:29:57.493109 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:57.493003    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 12 00:29:57.493109 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:57.493101    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/746054e5-684e-4c55-824b-6fdde562eb02-metrics-certs podName:746054e5-684e-4c55-824b-6fdde562eb02 nodeName:}" failed. No retries permitted until 2026-09-12 00:30:29.493052805 +0000 UTC m=+66.167111659 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/746054e5-684e-4c55-824b-6fdde562eb02-metrics-certs") pod "network-metrics-daemon-7tkvx" (UID: "746054e5-684e-4c55-824b-6fdde562eb02") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 12 00:29:57.493281 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:57.493122    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed1e49a7-a9ae-4e55-8a01-c88326238a56-original-pull-secret podName:ed1e49a7-a9ae-4e55-8a01-c88326238a56 nodeName:}" failed. No retries permitted until 2026-09-12 00:30:29.493111152 +0000 UTC m=+66.167169989 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ed1e49a7-a9ae-4e55-8a01-c88326238a56-original-pull-secret") pod "global-pull-secret-syncer-b2hjw" (UID: "ed1e49a7-a9ae-4e55-8a01-c88326238a56") : object "kube-system"/"original-pull-secret" not registered
Sep 12 00:29:57.594262 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:57.594229    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwrlm\" (UniqueName: \"kubernetes.io/projected/550df110-4473-4487-b318-d72d9babcaa3-kube-api-access-kwrlm\") pod \"network-check-target-f6t44\" (UID: \"550df110-4473-4487-b318-d72d9babcaa3\") " pod="openshift-network-diagnostics/network-check-target-f6t44"
Sep 12 00:29:57.594444 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:57.594403    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 12 00:29:57.594444 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:57.594428    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 12 00:29:57.594444 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:57.594439    2558 projected.go:196] Error preparing data for projected volume kube-api-access-kwrlm for pod openshift-network-diagnostics/network-check-target-f6t44: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:29:57.594569 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:57.594501    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/550df110-4473-4487-b318-d72d9babcaa3-kube-api-access-kwrlm podName:550df110-4473-4487-b318-d72d9babcaa3 nodeName:}" failed. No retries permitted until 2026-09-12 00:30:29.594479642 +0000 UTC m=+66.268538499 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-kwrlm" (UniqueName: "kubernetes.io/projected/550df110-4473-4487-b318-d72d9babcaa3-kube-api-access-kwrlm") pod "network-check-target-f6t44" (UID: "550df110-4473-4487-b318-d72d9babcaa3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 12 00:29:58.135025 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.134995    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-85.ec2.internal" event="NodeReady"
Sep 12 00:29:58.135554 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.135151    2558 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 12 00:29:58.170646 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.170607    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-599f5d46cb-zk8ln"]
Sep 12 00:29:58.187976 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.187947    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5wzpq"]
Sep 12 00:29:58.188171 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.188080    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:29:58.190654 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.190632    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-cg9mt\""
Sep 12 00:29:58.191019 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.191001    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 12 00:29:58.191315 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.191296    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 12 00:29:58.191604 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.191584    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 12 00:29:58.196432 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.196404    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 12 00:29:58.210027 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.210009    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-vcclb"]
Sep 12 00:29:58.210185 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.210170    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5wzpq"
Sep 12 00:29:58.212864 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.212726    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 12 00:29:58.212864 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.212758    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-h7jvg\""
Sep 12 00:29:58.212864 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.212726    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 12 00:29:58.236186 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.236166    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-599f5d46cb-zk8ln"]
Sep 12 00:29:58.236296 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.236204    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vcclb"]
Sep 12 00:29:58.236296 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.236223    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5wzpq"]
Sep 12 00:29:58.236296 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.236277    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vcclb"
Sep 12 00:29:58.238895 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.238870    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 12 00:29:58.238983 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.238906    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 12 00:29:58.238983 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.238923    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zbz7v\""
Sep 12 00:29:58.239107 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.238955    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 12 00:29:58.300283 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.300199    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-tls\") pod \"image-registry-599f5d46cb-zk8ln\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") " pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:29:58.300283 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.300239    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-image-registry-private-configuration\") pod \"image-registry-599f5d46cb-zk8ln\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") " pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:29:58.300283 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.300260    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-installation-pull-secrets\") pod \"image-registry-599f5d46cb-zk8ln\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") " pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:29:58.300555 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.300355    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0b419ed-e351-402a-9b9b-ff1534048f68-metrics-tls\") pod \"dns-default-5wzpq\" (UID: \"f0b419ed-e351-402a-9b9b-ff1534048f68\") " pod="openshift-dns/dns-default-5wzpq"
Sep 12 00:29:58.300555 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.300395    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-certificates\") pod \"image-registry-599f5d46cb-zk8ln\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") " pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:29:58.300555 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.300424    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f0b419ed-e351-402a-9b9b-ff1534048f68-config-volume\") pod \"dns-default-5wzpq\" (UID: \"f0b419ed-e351-402a-9b9b-ff1534048f68\") " pod="openshift-dns/dns-default-5wzpq"
Sep 12 00:29:58.300555 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.300444    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0b419ed-e351-402a-9b9b-ff1534048f68-tmp-dir\") pod \"dns-default-5wzpq\" (UID: \"f0b419ed-e351-402a-9b9b-ff1534048f68\") " pod="openshift-dns/dns-default-5wzpq"
Sep 12 00:29:58.300555 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.300464    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj4lf\" (UniqueName: \"kubernetes.io/projected/f0b419ed-e351-402a-9b9b-ff1534048f68-kube-api-access-wj4lf\") pod \"dns-default-5wzpq\" (UID: \"f0b419ed-e351-402a-9b9b-ff1534048f68\") " pod="openshift-dns/dns-default-5wzpq"
Sep 12 00:29:58.300555 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.300531    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-trusted-ca\") pod \"image-registry-599f5d46cb-zk8ln\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") " pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:29:58.300783 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.300615    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-ca-trust-extracted\") pod \"image-registry-599f5d46cb-zk8ln\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") " pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:29:58.300783 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.300657    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-bound-sa-token\") pod \"image-registry-599f5d46cb-zk8ln\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") " pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:29:58.300783 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.300683    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rtmq\" (UniqueName: \"kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-kube-api-access-8rtmq\") pod \"image-registry-599f5d46cb-zk8ln\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") " pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:29:58.339326 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.339296    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:29:58.388444 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.388406    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:29:58.401365 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.401334    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-bound-sa-token\") pod \"image-registry-599f5d46cb-zk8ln\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") " pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:29:58.401527 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.401370    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rtmq\" (UniqueName: \"kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-kube-api-access-8rtmq\") pod \"image-registry-599f5d46cb-zk8ln\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") " pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:29:58.401527 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.401402    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/421f63c6-8abc-4091-a546-2754a4f2066f-cert\") pod \"ingress-canary-vcclb\" (UID: \"421f63c6-8abc-4091-a546-2754a4f2066f\") " pod="openshift-ingress-canary/ingress-canary-vcclb"
Sep 12 00:29:58.401527 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.401428    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-tls\") pod \"image-registry-599f5d46cb-zk8ln\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") " pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:29:58.401527 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.401479    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-image-registry-private-configuration\") pod \"image-registry-599f5d46cb-zk8ln\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") " pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:29:58.401527 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.401501    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-installation-pull-secrets\") pod \"image-registry-599f5d46cb-zk8ln\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") " pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:29:58.401527 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:58.401509    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 12 00:29:58.401527 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:58.401528    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-599f5d46cb-zk8ln: secret "image-registry-tls" not found
Sep 12 00:29:58.401864 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.401564    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0b419ed-e351-402a-9b9b-ff1534048f68-metrics-tls\") pod \"dns-default-5wzpq\" (UID: \"f0b419ed-e351-402a-9b9b-ff1534048f68\") " pod="openshift-dns/dns-default-5wzpq"
Sep 12 00:29:58.401864 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:58.401583    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-tls podName:dbe5b08b-127a-4b6c-90db-7abd802f6e9a nodeName:}" failed. No retries permitted until 2026-09-12 00:29:58.90156377 +0000 UTC m=+35.575622617 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-tls") pod "image-registry-599f5d46cb-zk8ln" (UID: "dbe5b08b-127a-4b6c-90db-7abd802f6e9a") : secret "image-registry-tls" not found
Sep 12 00:29:58.401864 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.401609    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-certificates\") pod \"image-registry-599f5d46cb-zk8ln\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") " pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:29:58.401864 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:58.401671    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 12 00:29:58.401864 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.401692    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f0b419ed-e351-402a-9b9b-ff1534048f68-config-volume\") pod \"dns-default-5wzpq\" (UID: \"f0b419ed-e351-402a-9b9b-ff1534048f68\") " pod="openshift-dns/dns-default-5wzpq"
Sep 12 00:29:58.401864 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.401721    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0b419ed-e351-402a-9b9b-ff1534048f68-tmp-dir\") pod \"dns-default-5wzpq\" (UID: \"f0b419ed-e351-402a-9b9b-ff1534048f68\") " pod="openshift-dns/dns-default-5wzpq"
Sep 12 00:29:58.401864 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.401745    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj4lf\" (UniqueName: \"kubernetes.io/projected/f0b419ed-e351-402a-9b9b-ff1534048f68-kube-api-access-wj4lf\") pod \"dns-default-5wzpq\" (UID: \"f0b419ed-e351-402a-9b9b-ff1534048f68\") " pod="openshift-dns/dns-default-5wzpq"
Sep 12 00:29:58.401864 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.401785    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-trusted-ca\") pod \"image-registry-599f5d46cb-zk8ln\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") " pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:29:58.401864 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.401818    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2flfx\" (UniqueName: \"kubernetes.io/projected/421f63c6-8abc-4091-a546-2754a4f2066f-kube-api-access-2flfx\") pod \"ingress-canary-vcclb\" (UID: \"421f63c6-8abc-4091-a546-2754a4f2066f\") " pod="openshift-ingress-canary/ingress-canary-vcclb"
Sep 12 00:29:58.401864 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.401855    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-ca-trust-extracted\") pod \"image-registry-599f5d46cb-zk8ln\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") " pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:29:58.402303 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.402005    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0b419ed-e351-402a-9b9b-ff1534048f68-tmp-dir\") pod \"dns-default-5wzpq\" (UID: \"f0b419ed-e351-402a-9b9b-ff1534048f68\") " pod="openshift-dns/dns-default-5wzpq"
Sep 12 00:29:58.402303 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:58.402049    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0b419ed-e351-402a-9b9b-ff1534048f68-metrics-tls podName:f0b419ed-e351-402a-9b9b-ff1534048f68 nodeName:}" failed. No retries permitted until 2026-09-12 00:29:58.9020324 +0000 UTC m=+35.576091250 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f0b419ed-e351-402a-9b9b-ff1534048f68-metrics-tls") pod "dns-default-5wzpq" (UID: "f0b419ed-e351-402a-9b9b-ff1534048f68") : secret "dns-default-metrics-tls" not found
Sep 12 00:29:58.402303 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.402269    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-ca-trust-extracted\") pod \"image-registry-599f5d46cb-zk8ln\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") " pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:29:58.402303 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.402290    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-certificates\") pod \"image-registry-599f5d46cb-zk8ln\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") " pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:29:58.402454 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.402292    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f0b419ed-e351-402a-9b9b-ff1534048f68-config-volume\") pod \"dns-default-5wzpq\" (UID: \"f0b419ed-e351-402a-9b9b-ff1534048f68\") " pod="openshift-dns/dns-default-5wzpq"
Sep 12 00:29:58.403223 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.403206    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-trusted-ca\") pod \"image-registry-599f5d46cb-zk8ln\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") " pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:29:58.405887 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.405868    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-image-registry-private-configuration\") pod \"image-registry-599f5d46cb-zk8ln\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") " pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:29:58.405931 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.405920    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-installation-pull-secrets\") pod \"image-registry-599f5d46cb-zk8ln\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") " pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:29:58.409829 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.409803    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-bound-sa-token\") pod \"image-registry-599f5d46cb-zk8ln\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") " pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:29:58.409939 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.409897    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rtmq\" (UniqueName: \"kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-kube-api-access-8rtmq\") pod \"image-registry-599f5d46cb-zk8ln\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") " pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:29:58.410320 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.410287    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj4lf\" (UniqueName: \"kubernetes.io/projected/f0b419ed-e351-402a-9b9b-ff1534048f68-kube-api-access-wj4lf\") pod \"dns-default-5wzpq\" (UID: \"f0b419ed-e351-402a-9b9b-ff1534048f68\") " pod="openshift-dns/dns-default-5wzpq"
Sep 12 00:29:58.502731 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.502694    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/421f63c6-8abc-4091-a546-2754a4f2066f-cert\") pod \"ingress-canary-vcclb\" (UID: \"421f63c6-8abc-4091-a546-2754a4f2066f\") " pod="openshift-ingress-canary/ingress-canary-vcclb"
Sep 12 00:29:58.502901 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.502817    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2flfx\" (UniqueName: \"kubernetes.io/projected/421f63c6-8abc-4091-a546-2754a4f2066f-kube-api-access-2flfx\") pod \"ingress-canary-vcclb\" (UID: \"421f63c6-8abc-4091-a546-2754a4f2066f\") " pod="openshift-ingress-canary/ingress-canary-vcclb"
Sep 12 00:29:58.505610 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.505583    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/421f63c6-8abc-4091-a546-2754a4f2066f-cert\") pod \"ingress-canary-vcclb\" (UID: \"421f63c6-8abc-4091-a546-2754a4f2066f\") " pod="openshift-ingress-canary/ingress-canary-vcclb"
Sep 12 00:29:58.511013 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.510985    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2flfx\" (UniqueName: \"kubernetes.io/projected/421f63c6-8abc-4091-a546-2754a4f2066f-kube-api-access-2flfx\") pod \"ingress-canary-vcclb\" (UID: \"421f63c6-8abc-4091-a546-2754a4f2066f\") " pod="openshift-ingress-canary/ingress-canary-vcclb"
Sep 12 00:29:58.545407 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.545378    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vcclb"
Sep 12 00:29:58.731846 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.731606    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:29:58.732017 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.731618    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f6t44"
Sep 12 00:29:58.732101 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.731674    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:29:58.734627 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.734605    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 12 00:29:58.734627 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.734605    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 12 00:29:58.734816 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.734649    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 12 00:29:58.734816 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.734669    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9g5b8\""
Sep 12 00:29:58.734816 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.734611    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 12 00:29:58.735664 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.735644    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8bjlg\""
Sep 12 00:29:58.872602 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.872520    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vcclb"]
Sep 12 00:29:58.877528 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:29:58.877498    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod421f63c6_8abc_4091_a546_2754a4f2066f.slice/crio-1a5599024469e96fdf00215d64fdca09d2d41e8fa7a77ae3d58b461a0f7f6e6f WatchSource:0}: Error finding container 1a5599024469e96fdf00215d64fdca09d2d41e8fa7a77ae3d58b461a0f7f6e6f: Status 404 returned error can't find the container with id 1a5599024469e96fdf00215d64fdca09d2d41e8fa7a77ae3d58b461a0f7f6e6f
Sep 12 00:29:58.905959 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.905908    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vcclb" event={"ID":"421f63c6-8abc-4091-a546-2754a4f2066f","Type":"ContainerStarted","Data":"1a5599024469e96fdf00215d64fdca09d2d41e8fa7a77ae3d58b461a0f7f6e6f"}
Sep 12 00:29:58.906256 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.906235    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-tls\") pod \"image-registry-599f5d46cb-zk8ln\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") " pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:29:58.906351 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:58.906303    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0b419ed-e351-402a-9b9b-ff1534048f68-metrics-tls\") pod \"dns-default-5wzpq\" (UID: \"f0b419ed-e351-402a-9b9b-ff1534048f68\") " pod="openshift-dns/dns-default-5wzpq"
Sep 12 00:29:58.906415 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:58.906397    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 12 00:29:58.906415 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:58.906407    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 12 00:29:58.906501 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:58.906413    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-599f5d46cb-zk8ln: secret "image-registry-tls" not found
Sep 12 00:29:58.906501 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:58.906483    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-tls podName:dbe5b08b-127a-4b6c-90db-7abd802f6e9a nodeName:}" failed. No retries permitted until 2026-09-12 00:29:59.906466116 +0000 UTC m=+36.580524955 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-tls") pod "image-registry-599f5d46cb-zk8ln" (UID: "dbe5b08b-127a-4b6c-90db-7abd802f6e9a") : secret "image-registry-tls" not found
Sep 12 00:29:58.906501 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:58.906496    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0b419ed-e351-402a-9b9b-ff1534048f68-metrics-tls podName:f0b419ed-e351-402a-9b9b-ff1534048f68 nodeName:}" failed. No retries permitted until 2026-09-12 00:29:59.906490168 +0000 UTC m=+36.580549005 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f0b419ed-e351-402a-9b9b-ff1534048f68-metrics-tls") pod "dns-default-5wzpq" (UID: "f0b419ed-e351-402a-9b9b-ff1534048f68") : secret "dns-default-metrics-tls" not found
Sep 12 00:29:59.914027 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:59.913341    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0b419ed-e351-402a-9b9b-ff1534048f68-metrics-tls\") pod \"dns-default-5wzpq\" (UID: \"f0b419ed-e351-402a-9b9b-ff1534048f68\") " pod="openshift-dns/dns-default-5wzpq"
Sep 12 00:29:59.914027 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:29:59.913435    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-tls\") pod \"image-registry-599f5d46cb-zk8ln\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") " pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:29:59.914027 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:59.913557    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 12 00:29:59.914027 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:59.913569    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-599f5d46cb-zk8ln: secret "image-registry-tls" not found
Sep 12 00:29:59.914027 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:59.913617    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-tls podName:dbe5b08b-127a-4b6c-90db-7abd802f6e9a nodeName:}" failed. No retries permitted until 2026-09-12 00:30:01.913599544 +0000 UTC m=+38.587658382 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-tls") pod "image-registry-599f5d46cb-zk8ln" (UID: "dbe5b08b-127a-4b6c-90db-7abd802f6e9a") : secret "image-registry-tls" not found
Sep 12 00:29:59.914027 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:59.913940    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 12 00:29:59.914027 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:29:59.913980    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0b419ed-e351-402a-9b9b-ff1534048f68-metrics-tls podName:f0b419ed-e351-402a-9b9b-ff1534048f68 nodeName:}" failed. No retries permitted until 2026-09-12 00:30:01.913965632 +0000 UTC m=+38.588024483 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f0b419ed-e351-402a-9b9b-ff1534048f68-metrics-tls") pod "dns-default-5wzpq" (UID: "f0b419ed-e351-402a-9b9b-ff1534048f68") : secret "dns-default-metrics-tls" not found
Sep 12 00:30:01.930503 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:01.930471    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-tls\") pod \"image-registry-599f5d46cb-zk8ln\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") " pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:30:01.931034 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:01.930530    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0b419ed-e351-402a-9b9b-ff1534048f68-metrics-tls\") pod \"dns-default-5wzpq\" (UID: \"f0b419ed-e351-402a-9b9b-ff1534048f68\") " pod="openshift-dns/dns-default-5wzpq"
Sep 12 00:30:01.931034 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:30:01.930628    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 12 00:30:01.931034 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:30:01.930635    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 12 00:30:01.931034 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:30:01.930651    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-599f5d46cb-zk8ln: secret "image-registry-tls" not found
Sep 12 00:30:01.931034 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:30:01.930677    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0b419ed-e351-402a-9b9b-ff1534048f68-metrics-tls podName:f0b419ed-e351-402a-9b9b-ff1534048f68 nodeName:}" failed. No retries permitted until 2026-09-12 00:30:05.930664638 +0000 UTC m=+42.604723494 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f0b419ed-e351-402a-9b9b-ff1534048f68-metrics-tls") pod "dns-default-5wzpq" (UID: "f0b419ed-e351-402a-9b9b-ff1534048f68") : secret "dns-default-metrics-tls" not found
Sep 12 00:30:01.931034 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:30:01.930698    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-tls podName:dbe5b08b-127a-4b6c-90db-7abd802f6e9a nodeName:}" failed. No retries permitted until 2026-09-12 00:30:05.930684244 +0000 UTC m=+42.604743111 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-tls") pod "image-registry-599f5d46cb-zk8ln" (UID: "dbe5b08b-127a-4b6c-90db-7abd802f6e9a") : secret "image-registry-tls" not found
Sep 12 00:30:02.441292 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:02.441251    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:30:02.580381 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:02.580343    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:30:02.923050 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:02.922964    2558 generic.go:356] "Generic (PLEG): container finished" podID="ca5a34af-39bd-486b-8293-387ff1f949e2" containerID="80f4eb48cd9e2c231eb188b8d447eda1c0b33f323dd4a81b667ef71a53728821" exitCode=0
Sep 12 00:30:02.923050 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:02.923032    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m6mrh" event={"ID":"ca5a34af-39bd-486b-8293-387ff1f949e2","Type":"ContainerDied","Data":"80f4eb48cd9e2c231eb188b8d447eda1c0b33f323dd4a81b667ef71a53728821"}
Sep 12 00:30:02.924528 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:02.924504    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vcclb" event={"ID":"421f63c6-8abc-4091-a546-2754a4f2066f","Type":"ContainerStarted","Data":"43e7940af6e28f61455c9bcfc8b0535f905eaf9501e96fb0568518ef19710726"}
Sep 12 00:30:03.455000 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:03.454816    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:30:03.502554 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:03.502524    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:30:03.929724 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:03.929640    2558 generic.go:356] "Generic (PLEG): container finished" podID="ca5a34af-39bd-486b-8293-387ff1f949e2" containerID="6bf08ade0babdeb8b24f3b785b503f14fd11b41632656eb6bb83de9eb5642383" exitCode=0
Sep 12 00:30:03.929851 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:03.929723    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m6mrh" event={"ID":"ca5a34af-39bd-486b-8293-387ff1f949e2","Type":"ContainerDied","Data":"6bf08ade0babdeb8b24f3b785b503f14fd11b41632656eb6bb83de9eb5642383"}
Sep 12 00:30:03.949999 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:03.949953    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-vcclb" podStartSLOduration=2.688726138 podStartE2EDuration="5.949936914s" totalImagesPullingTime="3.261210776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:29:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:30:02.956313339 +0000 UTC m=+39.630372193" watchObservedRunningTime="2026-09-12 00:30:03.949936914 +0000 UTC m=+40.623995764"
Sep 12 00:30:04.710457 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:04.710427    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:30:04.806752 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:04.806721    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:30:04.935329 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:04.935295    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m6mrh" event={"ID":"ca5a34af-39bd-486b-8293-387ff1f949e2","Type":"ContainerStarted","Data":"87a2ee391a3eeceaa3b14c3d742e400b9b2ed09bc9e274c97bcbcb59b30cc84d"}
Sep 12 00:30:04.956580 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:04.956531    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-m6mrh" podStartSLOduration=9.011901871 podStartE2EDuration="40.956515393s" totalImagesPullingTime="31.944613522s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:29:24 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:30:04.954868204 +0000 UTC m=+41.628927076" watchObservedRunningTime="2026-09-12 00:30:04.956515393 +0000 UTC m=+41.630574252"
Sep 12 00:30:05.073671 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:05.073597    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:30:05.175222 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:05.175189    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:30:05.795759 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:05.795729    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:30:05.846023 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:05.845993    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:30:05.963664 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:05.963629    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-tls\") pod \"image-registry-599f5d46cb-zk8ln\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") " pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:30:05.963842 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:05.963698    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0b419ed-e351-402a-9b9b-ff1534048f68-metrics-tls\") pod \"dns-default-5wzpq\" (UID: \"f0b419ed-e351-402a-9b9b-ff1534048f68\") " pod="openshift-dns/dns-default-5wzpq"
Sep 12 00:30:05.963842 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:30:05.963770    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 12 00:30:05.963842 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:30:05.963787    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-599f5d46cb-zk8ln: secret "image-registry-tls" not found
Sep 12 00:30:05.963842 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:30:05.963813    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 12 00:30:05.963842 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:30:05.963840    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-tls podName:dbe5b08b-127a-4b6c-90db-7abd802f6e9a nodeName:}" failed. No retries permitted until 2026-09-12 00:30:13.963826333 +0000 UTC m=+50.637885169 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-tls") pod "image-registry-599f5d46cb-zk8ln" (UID: "dbe5b08b-127a-4b6c-90db-7abd802f6e9a") : secret "image-registry-tls" not found
Sep 12 00:30:05.964020 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:30:05.963861    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0b419ed-e351-402a-9b9b-ff1534048f68-metrics-tls podName:f0b419ed-e351-402a-9b9b-ff1534048f68 nodeName:}" failed. No retries permitted until 2026-09-12 00:30:13.963845951 +0000 UTC m=+50.637904789 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f0b419ed-e351-402a-9b9b-ff1534048f68-metrics-tls") pod "dns-default-5wzpq" (UID: "f0b419ed-e351-402a-9b9b-ff1534048f68") : secret "dns-default-metrics-tls" not found
Sep 12 00:30:06.042165 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:06.042133    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:30:06.141177 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:06.141097    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:30:06.241357 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:06.241323    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:30:14.020560 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:14.020511    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-tls\") pod \"image-registry-599f5d46cb-zk8ln\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") " pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:30:14.021026 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:14.020589    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0b419ed-e351-402a-9b9b-ff1534048f68-metrics-tls\") pod \"dns-default-5wzpq\" (UID: \"f0b419ed-e351-402a-9b9b-ff1534048f68\") " pod="openshift-dns/dns-default-5wzpq"
Sep 12 00:30:14.021026 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:30:14.020655    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 12 00:30:14.021026 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:30:14.020673    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-599f5d46cb-zk8ln: secret "image-registry-tls" not found
Sep 12 00:30:14.021026 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:30:14.020689    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 12 00:30:14.021026 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:30:14.020723    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-tls podName:dbe5b08b-127a-4b6c-90db-7abd802f6e9a nodeName:}" failed. No retries permitted until 2026-09-12 00:30:30.020708663 +0000 UTC m=+66.694767499 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-tls") pod "image-registry-599f5d46cb-zk8ln" (UID: "dbe5b08b-127a-4b6c-90db-7abd802f6e9a") : secret "image-registry-tls" not found
Sep 12 00:30:14.021026 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:30:14.020740    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0b419ed-e351-402a-9b9b-ff1534048f68-metrics-tls podName:f0b419ed-e351-402a-9b9b-ff1534048f68 nodeName:}" failed. No retries permitted until 2026-09-12 00:30:30.020728642 +0000 UTC m=+66.694787493 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f0b419ed-e351-402a-9b9b-ff1534048f68-metrics-tls") pod "dns-default-5wzpq" (UID: "f0b419ed-e351-402a-9b9b-ff1534048f68") : secret "dns-default-metrics-tls" not found
Sep 12 00:30:24.902927 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:24.902899    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cmk6p"
Sep 12 00:30:25.047099 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:25.047046    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:30:25.108165 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:25.108138    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:30:27.643378 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:27.643345    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:30:27.694938 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:27.694911    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:30:28.052791 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:28.052758    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:30:29.534107 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:29.534047    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ed1e49a7-a9ae-4e55-8a01-c88326238a56-original-pull-secret\") pod \"global-pull-secret-syncer-b2hjw\" (UID: \"ed1e49a7-a9ae-4e55-8a01-c88326238a56\") " pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:30:29.534604 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:29.534115    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/746054e5-684e-4c55-824b-6fdde562eb02-metrics-certs\") pod \"network-metrics-daemon-7tkvx\" (UID: \"746054e5-684e-4c55-824b-6fdde562eb02\") " pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:30:29.536679 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:29.536659    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 12 00:30:29.536828 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:29.536811    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 12 00:30:29.544963 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:30:29.544944    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 12 00:30:29.545034 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:30:29.545005    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/746054e5-684e-4c55-824b-6fdde562eb02-metrics-certs podName:746054e5-684e-4c55-824b-6fdde562eb02 nodeName:}" failed. No retries permitted until 2026-09-12 00:31:33.544988824 +0000 UTC m=+130.219047674 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/746054e5-684e-4c55-824b-6fdde562eb02-metrics-certs") pod "network-metrics-daemon-7tkvx" (UID: "746054e5-684e-4c55-824b-6fdde562eb02") : secret "metrics-daemon-secret" not found
Sep 12 00:30:29.548025 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:29.548003    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ed1e49a7-a9ae-4e55-8a01-c88326238a56-original-pull-secret\") pod \"global-pull-secret-syncer-b2hjw\" (UID: \"ed1e49a7-a9ae-4e55-8a01-c88326238a56\") " pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:30:29.634483 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:29.634426    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwrlm\" (UniqueName: \"kubernetes.io/projected/550df110-4473-4487-b318-d72d9babcaa3-kube-api-access-kwrlm\") pod \"network-check-target-f6t44\" (UID: \"550df110-4473-4487-b318-d72d9babcaa3\") " pod="openshift-network-diagnostics/network-check-target-f6t44"
Sep 12 00:30:29.637669 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:29.637647    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 12 00:30:29.646755 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:29.646730    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2hjw"
Sep 12 00:30:29.648080 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:29.648061    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 12 00:30:29.658574 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:29.658549    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwrlm\" (UniqueName: \"kubernetes.io/projected/550df110-4473-4487-b318-d72d9babcaa3-kube-api-access-kwrlm\") pod \"network-check-target-f6t44\" (UID: \"550df110-4473-4487-b318-d72d9babcaa3\") " pod="openshift-network-diagnostics/network-check-target-f6t44"
Sep 12 00:30:29.912798 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:29.912717    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-b2hjw"]
Sep 12 00:30:29.915966 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:30:29.915935    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded1e49a7_a9ae_4e55_8a01_c88326238a56.slice/crio-ff7e19cf71fc309665d49b503281e4b9c1a8177f9c967f18aae5749b83c5bedb WatchSource:0}: Error finding container ff7e19cf71fc309665d49b503281e4b9c1a8177f9c967f18aae5749b83c5bedb: Status 404 returned error can't find the container with id ff7e19cf71fc309665d49b503281e4b9c1a8177f9c967f18aae5749b83c5bedb
Sep 12 00:30:29.955867 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:29.955841    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9g5b8\""
Sep 12 00:30:29.963838 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:29.963817    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f6t44"
Sep 12 00:30:29.993143 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:29.993115    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-b2hjw" event={"ID":"ed1e49a7-a9ae-4e55-8a01-c88326238a56","Type":"ContainerStarted","Data":"ff7e19cf71fc309665d49b503281e4b9c1a8177f9c967f18aae5749b83c5bedb"}
Sep 12 00:30:30.037730 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:30.037699    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-tls\") pod \"image-registry-599f5d46cb-zk8ln\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") " pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:30:30.037885 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:30.037777    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0b419ed-e351-402a-9b9b-ff1534048f68-metrics-tls\") pod \"dns-default-5wzpq\" (UID: \"f0b419ed-e351-402a-9b9b-ff1534048f68\") " pod="openshift-dns/dns-default-5wzpq"
Sep 12 00:30:30.037885 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:30:30.037847    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 12 00:30:30.037885 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:30:30.037873    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-599f5d46cb-zk8ln: secret "image-registry-tls" not found
Sep 12 00:30:30.037996 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:30:30.037885    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 12 00:30:30.037996 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:30:30.037945    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0b419ed-e351-402a-9b9b-ff1534048f68-metrics-tls podName:f0b419ed-e351-402a-9b9b-ff1534048f68 nodeName:}" failed. No retries permitted until 2026-09-12 00:31:02.03792661 +0000 UTC m=+98.711985464 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f0b419ed-e351-402a-9b9b-ff1534048f68-metrics-tls") pod "dns-default-5wzpq" (UID: "f0b419ed-e351-402a-9b9b-ff1534048f68") : secret "dns-default-metrics-tls" not found
Sep 12 00:30:30.037996 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:30:30.037963    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-tls podName:dbe5b08b-127a-4b6c-90db-7abd802f6e9a nodeName:}" failed. No retries permitted until 2026-09-12 00:31:02.037952513 +0000 UTC m=+98.712011357 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-tls") pod "image-registry-599f5d46cb-zk8ln" (UID: "dbe5b08b-127a-4b6c-90db-7abd802f6e9a") : secret "image-registry-tls" not found
Sep 12 00:30:30.233858 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:30.233826    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-f6t44"]
Sep 12 00:30:30.236755 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:30:30.236725    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod550df110_4473_4487_b318_d72d9babcaa3.slice/crio-4007436570d9ef86b567d8c55cf2840667bceb249fce4b077996976f97323019 WatchSource:0}: Error finding container 4007436570d9ef86b567d8c55cf2840667bceb249fce4b077996976f97323019: Status 404 returned error can't find the container with id 4007436570d9ef86b567d8c55cf2840667bceb249fce4b077996976f97323019
Sep 12 00:30:30.998101 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:30.998045    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-f6t44" event={"ID":"550df110-4473-4487-b318-d72d9babcaa3","Type":"ContainerStarted","Data":"4007436570d9ef86b567d8c55cf2840667bceb249fce4b077996976f97323019"}
Sep 12 00:30:35.009466 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:35.009377    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-f6t44" event={"ID":"550df110-4473-4487-b318-d72d9babcaa3","Type":"ContainerStarted","Data":"223784c0393ba15343265f63fe1838af2469fca53948178c751a632c42bae562"}
Sep 12 00:30:35.009960 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:35.009475    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-f6t44"
Sep 12 00:30:35.010845 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:35.010822    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-b2hjw" event={"ID":"ed1e49a7-a9ae-4e55-8a01-c88326238a56","Type":"ContainerStarted","Data":"f23ec21285a7bfd4f82516559ab295c325c22fffc757a221c25e34bf99132876"}
Sep 12 00:30:35.023790 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:35.023745    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-f6t44" podStartSLOduration=66.566064106 podStartE2EDuration="1m11.023732194s" totalImagesPullingTime="4.457668088s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:29:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:30:35.022925217 +0000 UTC m=+71.696984075" watchObservedRunningTime="2026-09-12 00:30:35.023732194 +0000 UTC m=+71.697791052"
Sep 12 00:30:56.435609 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:56.435574    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:30:56.483786 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:30:56.483759    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:31:02.058830 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:31:02.058782    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-tls\") pod \"image-registry-599f5d46cb-zk8ln\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") " pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:31:02.059317 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:31:02.058846    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0b419ed-e351-402a-9b9b-ff1534048f68-metrics-tls\") pod \"dns-default-5wzpq\" (UID: \"f0b419ed-e351-402a-9b9b-ff1534048f68\") " pod="openshift-dns/dns-default-5wzpq"
Sep 12 00:31:02.059317 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:31:02.058962    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 12 00:31:02.059317 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:31:02.058968    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 12 00:31:02.059317 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:31:02.058982    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-599f5d46cb-zk8ln: secret "image-registry-tls" not found
Sep 12 00:31:02.059317 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:31:02.059033    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0b419ed-e351-402a-9b9b-ff1534048f68-metrics-tls podName:f0b419ed-e351-402a-9b9b-ff1534048f68 nodeName:}" failed. No retries permitted until 2026-09-12 00:32:06.059014147 +0000 UTC m=+162.733072985 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f0b419ed-e351-402a-9b9b-ff1534048f68-metrics-tls") pod "dns-default-5wzpq" (UID: "f0b419ed-e351-402a-9b9b-ff1534048f68") : secret "dns-default-metrics-tls" not found
Sep 12 00:31:02.059317 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:31:02.059046    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-tls podName:dbe5b08b-127a-4b6c-90db-7abd802f6e9a nodeName:}" failed. No retries permitted until 2026-09-12 00:32:06.059040614 +0000 UTC m=+162.733099451 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-tls") pod "image-registry-599f5d46cb-zk8ln" (UID: "dbe5b08b-127a-4b6c-90db-7abd802f6e9a") : secret "image-registry-tls" not found
Sep 12 00:31:06.015579 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:31:06.015543    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-f6t44"
Sep 12 00:31:06.030527 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:31:06.030471    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-b2hjw" podStartSLOduration=97.248152949 podStartE2EDuration="1m42.030456664s" totalImagesPullingTime="4.782303715s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:29:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:30:35.035204524 +0000 UTC m=+71.709263393" watchObservedRunningTime="2026-09-12 00:31:06.030456664 +0000 UTC m=+102.704515522"
Sep 12 00:31:09.427016 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:31:09.426976    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f559bb96b-5kgb5"]
Sep 12 00:31:09.429979 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:31:09.429957    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f559bb96b-5kgb5"
Sep 12 00:31:09.434783 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:31:09.434761    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 12 00:31:09.434877 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:31:09.434801    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 12 00:31:09.435978 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:31:09.435953    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 12 00:31:09.436072 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:31:09.436047    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 12 00:31:09.440787 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:31:09.440767    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f559bb96b-5kgb5"]
Sep 12 00:31:09.508726 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:31:09.508694    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jh92\" (UniqueName: \"kubernetes.io/projected/ad14fec3-e51f-4da9-802d-bf7d32815e89-kube-api-access-2jh92\") pod \"klusterlet-addon-workmgr-6f559bb96b-5kgb5\" (UID: \"ad14fec3-e51f-4da9-802d-bf7d32815e89\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f559bb96b-5kgb5"
Sep 12 00:31:09.508726 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:31:09.508736    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ad14fec3-e51f-4da9-802d-bf7d32815e89-klusterlet-config\") pod \"klusterlet-addon-workmgr-6f559bb96b-5kgb5\" (UID: \"ad14fec3-e51f-4da9-802d-bf7d32815e89\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f559bb96b-5kgb5"
Sep 12 00:31:09.508970 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:31:09.508762    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad14fec3-e51f-4da9-802d-bf7d32815e89-tmp\") pod \"klusterlet-addon-workmgr-6f559bb96b-5kgb5\" (UID: \"ad14fec3-e51f-4da9-802d-bf7d32815e89\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f559bb96b-5kgb5"
Sep 12 00:31:09.609370 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:31:09.609325    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ad14fec3-e51f-4da9-802d-bf7d32815e89-klusterlet-config\") pod \"klusterlet-addon-workmgr-6f559bb96b-5kgb5\" (UID: \"ad14fec3-e51f-4da9-802d-bf7d32815e89\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f559bb96b-5kgb5"
Sep 12 00:31:09.609370 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:31:09.609382    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad14fec3-e51f-4da9-802d-bf7d32815e89-tmp\") pod \"klusterlet-addon-workmgr-6f559bb96b-5kgb5\" (UID: \"ad14fec3-e51f-4da9-802d-bf7d32815e89\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f559bb96b-5kgb5"
Sep 12 00:31:09.609597 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:31:09.609467    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jh92\" (UniqueName: \"kubernetes.io/projected/ad14fec3-e51f-4da9-802d-bf7d32815e89-kube-api-access-2jh92\") pod \"klusterlet-addon-workmgr-6f559bb96b-5kgb5\" (UID: \"ad14fec3-e51f-4da9-802d-bf7d32815e89\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f559bb96b-5kgb5"
Sep 12 00:31:09.609791 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:31:09.609775    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad14fec3-e51f-4da9-802d-bf7d32815e89-tmp\") pod \"klusterlet-addon-workmgr-6f559bb96b-5kgb5\" (UID: \"ad14fec3-e51f-4da9-802d-bf7d32815e89\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f559bb96b-5kgb5"
Sep 12 00:31:09.611962 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:31:09.611947    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ad14fec3-e51f-4da9-802d-bf7d32815e89-klusterlet-config\") pod \"klusterlet-addon-workmgr-6f559bb96b-5kgb5\" (UID: \"ad14fec3-e51f-4da9-802d-bf7d32815e89\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f559bb96b-5kgb5"
Sep 12 00:31:09.617365 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:31:09.617341    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jh92\" (UniqueName: \"kubernetes.io/projected/ad14fec3-e51f-4da9-802d-bf7d32815e89-kube-api-access-2jh92\") pod \"klusterlet-addon-workmgr-6f559bb96b-5kgb5\" (UID: \"ad14fec3-e51f-4da9-802d-bf7d32815e89\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f559bb96b-5kgb5"
Sep 12 00:31:09.738444 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:31:09.738419    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f559bb96b-5kgb5"
Sep 12 00:31:10.003344 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:31:10.003271    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f559bb96b-5kgb5"]
Sep 12 00:31:10.006378 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:31:10.006343    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad14fec3_e51f_4da9_802d_bf7d32815e89.slice/crio-1a8e1fc23d0ebe2fc15e421e784bf00126fe8bd28864fedd06644803ea00cf7a WatchSource:0}: Error finding container 1a8e1fc23d0ebe2fc15e421e784bf00126fe8bd28864fedd06644803ea00cf7a: Status 404 returned error can't find the container with id 1a8e1fc23d0ebe2fc15e421e784bf00126fe8bd28864fedd06644803ea00cf7a
Sep 12 00:31:10.093577 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:31:10.093545    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f559bb96b-5kgb5" event={"ID":"ad14fec3-e51f-4da9-802d-bf7d32815e89","Type":"ContainerStarted","Data":"1a8e1fc23d0ebe2fc15e421e784bf00126fe8bd28864fedd06644803ea00cf7a"}
Sep 12 00:31:14.107143 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:31:14.107037    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f559bb96b-5kgb5" event={"ID":"ad14fec3-e51f-4da9-802d-bf7d32815e89","Type":"ContainerStarted","Data":"9bca467ebf334a8240590959946b01ebd6d0d24d332e200fc53c8635ac8a9d28"}
Sep 12 00:31:14.107508 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:31:14.107267    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f559bb96b-5kgb5"
Sep 12 00:31:14.108807 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:31:14.108788    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f559bb96b-5kgb5"
Sep 12 00:31:14.121716 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:31:14.121672    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f559bb96b-5kgb5" podStartSLOduration=1.2788238619999999 podStartE2EDuration="5.121656192s" totalImagesPullingTime="3.84283233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:31:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:31:14.120874931 +0000 UTC m=+110.794933801" watchObservedRunningTime="2026-09-12 00:31:14.121656192 +0000 UTC m=+110.795715050"
Sep 12 00:31:33.587166 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:31:33.587113    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/746054e5-684e-4c55-824b-6fdde562eb02-metrics-certs\") pod \"network-metrics-daemon-7tkvx\" (UID: \"746054e5-684e-4c55-824b-6fdde562eb02\") " pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:31:33.587586 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:31:33.587247    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 12 00:31:33.587586 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:31:33.587327    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/746054e5-684e-4c55-824b-6fdde562eb02-metrics-certs podName:746054e5-684e-4c55-824b-6fdde562eb02 nodeName:}" failed. No retries permitted until 2026-09-12 00:33:35.587311158 +0000 UTC m=+252.261369995 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/746054e5-684e-4c55-824b-6fdde562eb02-metrics-certs") pod "network-metrics-daemon-7tkvx" (UID: "746054e5-684e-4c55-824b-6fdde562eb02") : secret "metrics-daemon-secret" not found
Sep 12 00:32:01.200598 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:32:01.200541    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln" podUID="dbe5b08b-127a-4b6c-90db-7abd802f6e9a"
Sep 12 00:32:01.219996 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:32:01.219962    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-5wzpq" podUID="f0b419ed-e351-402a-9b9b-ff1534048f68"
Sep 12 00:32:01.221370 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:32:01.221350    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:32:01.759418 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:32:01.759381    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-7tkvx" podUID="746054e5-684e-4c55-824b-6fdde562eb02"
Sep 12 00:32:02.224054 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:32:02.224027    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5wzpq"
Sep 12 00:32:06.117335 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:32:06.117294    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-tls\") pod \"image-registry-599f5d46cb-zk8ln\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") " pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:32:06.117844 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:32:06.117361    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0b419ed-e351-402a-9b9b-ff1534048f68-metrics-tls\") pod \"dns-default-5wzpq\" (UID: \"f0b419ed-e351-402a-9b9b-ff1534048f68\") " pod="openshift-dns/dns-default-5wzpq"
Sep 12 00:32:06.117844 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:32:06.117453    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 12 00:32:06.117844 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:32:06.117465    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 12 00:32:06.117844 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:32:06.117472    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-599f5d46cb-zk8ln: secret "image-registry-tls" not found
Sep 12 00:32:06.117844 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:32:06.117534    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0b419ed-e351-402a-9b9b-ff1534048f68-metrics-tls podName:f0b419ed-e351-402a-9b9b-ff1534048f68 nodeName:}" failed. No retries permitted until 2026-09-12 00:34:08.117516901 +0000 UTC m=+284.791575743 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f0b419ed-e351-402a-9b9b-ff1534048f68-metrics-tls") pod "dns-default-5wzpq" (UID: "f0b419ed-e351-402a-9b9b-ff1534048f68") : secret "dns-default-metrics-tls" not found
Sep 12 00:32:06.117844 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:32:06.117548    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-tls podName:dbe5b08b-127a-4b6c-90db-7abd802f6e9a nodeName:}" failed. No retries permitted until 2026-09-12 00:34:08.117542145 +0000 UTC m=+284.791600982 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-tls") pod "image-registry-599f5d46cb-zk8ln" (UID: "dbe5b08b-127a-4b6c-90db-7abd802f6e9a") : secret "image-registry-tls" not found
Sep 12 00:32:14.732293 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:32:14.732258    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:32:39.850149 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:32:39.850021    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:03.835740 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:03.835706    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:14.108341 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:14.108294    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f559bb96b-5kgb5" podUID="ad14fec3-e51f-4da9-802d-bf7d32815e89" containerName="acm-agent" probeResult="failure" output="Get \"http://10.133.0.9:8000/readyz\": dial tcp 10.133.0.9:8000: connect: connection refused"
Sep 12 00:33:14.396937 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:14.396854    2558 generic.go:356] "Generic (PLEG): container finished" podID="ad14fec3-e51f-4da9-802d-bf7d32815e89" containerID="9bca467ebf334a8240590959946b01ebd6d0d24d332e200fc53c8635ac8a9d28" exitCode=1
Sep 12 00:33:14.396937 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:14.396909    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f559bb96b-5kgb5" event={"ID":"ad14fec3-e51f-4da9-802d-bf7d32815e89","Type":"ContainerDied","Data":"9bca467ebf334a8240590959946b01ebd6d0d24d332e200fc53c8635ac8a9d28"}
Sep 12 00:33:14.397290 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:14.397274    2558 scope.go:117] "RemoveContainer" containerID="9bca467ebf334a8240590959946b01ebd6d0d24d332e200fc53c8635ac8a9d28"
Sep 12 00:33:15.402599 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:15.402561    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f559bb96b-5kgb5" event={"ID":"ad14fec3-e51f-4da9-802d-bf7d32815e89","Type":"ContainerStarted","Data":"4133de8583c1d92f03dabe0a573151543218f809b2d436da1d7b8cede9dc4663"}
Sep 12 00:33:15.403023 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:15.402852    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f559bb96b-5kgb5"
Sep 12 00:33:15.403420 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:15.403402    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f559bb96b-5kgb5"
Sep 12 00:33:28.986214 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:28.986179    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-d6d5fbc68-m67sb"]
Sep 12 00:33:28.989264 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:28.989248    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-cnml8"]
Sep 12 00:33:28.989398 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:28.989379    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-d6d5fbc68-m67sb"
Sep 12 00:33:28.991618 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:28.991594    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 12 00:33:28.991908 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:28.991891    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 12 00:33:28.992010 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:28.991989    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 12 00:33:28.992075 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:28.992018    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-nmn9k\""
Sep 12 00:33:28.992075 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:28.991998    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 12 00:33:28.992075 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:28.992058    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 12 00:33:28.992245 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:28.992154    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-cnml8"
Sep 12 00:33:28.992245 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:28.992155    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 12 00:33:28.994598 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:28.994581    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 12 00:33:28.994672 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:28.994606    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-f8gmb\""
Sep 12 00:33:28.994857 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:28.994841    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 12 00:33:28.995035 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:28.995019    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 12 00:33:28.995250 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:28.995235    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 12 00:33:29.000754 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.000730    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-cnml8"]
Sep 12 00:33:29.001435 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.001415    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 12 00:33:29.006978 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.006957    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-d6d5fbc68-m67sb"]
Sep 12 00:33:29.079911 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.079884    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-6wxf6"]
Sep 12 00:33:29.083695 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.083670    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-czf8h"]
Sep 12 00:33:29.084112 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.084064    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-6wxf6"
Sep 12 00:33:29.086990 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.086968    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 12 00:33:29.087205 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.087123    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 12 00:33:29.087205 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.087193    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 12 00:33:29.087320 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.087216    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-czf8h"
Sep 12 00:33:29.087320 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.087277    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 12 00:33:29.087604 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.087583    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-glvb5\""
Sep 12 00:33:29.089576 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.089561    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 12 00:33:29.089703 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.089567    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-l26g6\""
Sep 12 00:33:29.089915 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.089895    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 12 00:33:29.090000 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.089900    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 12 00:33:29.090239 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.090223    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 12 00:33:29.094375 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.094335    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 12 00:33:29.096624 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.096604    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-6wxf6"]
Sep 12 00:33:29.097470 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.097449    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-czf8h"]
Sep 12 00:33:29.106797 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.106777    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc47c0e2-02fe-408b-b0a5-5b046a81f1d7-service-ca-bundle\") pod \"insights-operator-5844b856db-cnml8\" (UID: \"fc47c0e2-02fe-408b-b0a5-5b046a81f1d7\") " pod="openshift-insights/insights-operator-5844b856db-cnml8"
Sep 12 00:33:29.106893 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.106821    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c47d77e0-78de-4d81-9e37-c9f5572e23a4-default-certificate\") pod \"router-default-d6d5fbc68-m67sb\" (UID: \"c47d77e0-78de-4d81-9e37-c9f5572e23a4\") " pod="openshift-ingress/router-default-d6d5fbc68-m67sb"
Sep 12 00:33:29.106893 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.106838    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c47d77e0-78de-4d81-9e37-c9f5572e23a4-service-ca-bundle\") pod \"router-default-d6d5fbc68-m67sb\" (UID: \"c47d77e0-78de-4d81-9e37-c9f5572e23a4\") " pod="openshift-ingress/router-default-d6d5fbc68-m67sb"
Sep 12 00:33:29.106893 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.106852    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c47d77e0-78de-4d81-9e37-c9f5572e23a4-stats-auth\") pod \"router-default-d6d5fbc68-m67sb\" (UID: \"c47d77e0-78de-4d81-9e37-c9f5572e23a4\") " pod="openshift-ingress/router-default-d6d5fbc68-m67sb"
Sep 12 00:33:29.107037 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.106933    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c47d77e0-78de-4d81-9e37-c9f5572e23a4-metrics-certs\") pod \"router-default-d6d5fbc68-m67sb\" (UID: \"c47d77e0-78de-4d81-9e37-c9f5572e23a4\") " pod="openshift-ingress/router-default-d6d5fbc68-m67sb"
Sep 12 00:33:29.107037 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.106960    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/fc47c0e2-02fe-408b-b0a5-5b046a81f1d7-snapshots\") pod \"insights-operator-5844b856db-cnml8\" (UID: \"fc47c0e2-02fe-408b-b0a5-5b046a81f1d7\") " pod="openshift-insights/insights-operator-5844b856db-cnml8"
Sep 12 00:33:29.107037 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.106989    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4f2z\" (UniqueName: \"kubernetes.io/projected/c47d77e0-78de-4d81-9e37-c9f5572e23a4-kube-api-access-f4f2z\") pod \"router-default-d6d5fbc68-m67sb\" (UID: \"c47d77e0-78de-4d81-9e37-c9f5572e23a4\") " pod="openshift-ingress/router-default-d6d5fbc68-m67sb"
Sep 12 00:33:29.107037 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.107005    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fc47c0e2-02fe-408b-b0a5-5b046a81f1d7-serving-cert\") pod \"insights-operator-5844b856db-cnml8\" (UID: \"fc47c0e2-02fe-408b-b0a5-5b046a81f1d7\") " pod="openshift-insights/insights-operator-5844b856db-cnml8"
Sep 12 00:33:29.107037 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.107023    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc47c0e2-02fe-408b-b0a5-5b046a81f1d7-trusted-ca-bundle\") pod \"insights-operator-5844b856db-cnml8\" (UID: \"fc47c0e2-02fe-408b-b0a5-5b046a81f1d7\") " pod="openshift-insights/insights-operator-5844b856db-cnml8"
Sep 12 00:33:29.107252 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.107071    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc47c0e2-02fe-408b-b0a5-5b046a81f1d7-tmp\") pod \"insights-operator-5844b856db-cnml8\" (UID: \"fc47c0e2-02fe-408b-b0a5-5b046a81f1d7\") " pod="openshift-insights/insights-operator-5844b856db-cnml8"
Sep 12 00:33:29.107252 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.107114    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4djk\" (UniqueName: \"kubernetes.io/projected/fc47c0e2-02fe-408b-b0a5-5b046a81f1d7-kube-api-access-b4djk\") pod \"insights-operator-5844b856db-cnml8\" (UID: \"fc47c0e2-02fe-408b-b0a5-5b046a81f1d7\") " pod="openshift-insights/insights-operator-5844b856db-cnml8"
Sep 12 00:33:29.126111 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.126038    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:29.176764 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.176736    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:29.208370 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.208347    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4djk\" (UniqueName: \"kubernetes.io/projected/fc47c0e2-02fe-408b-b0a5-5b046a81f1d7-kube-api-access-b4djk\") pod \"insights-operator-5844b856db-cnml8\" (UID: \"fc47c0e2-02fe-408b-b0a5-5b046a81f1d7\") " pod="openshift-insights/insights-operator-5844b856db-cnml8"
Sep 12 00:33:29.208463 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.208391    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdjvb\" (UniqueName: \"kubernetes.io/projected/3a305be2-8e7b-460c-adbf-691ae6c79683-kube-api-access-jdjvb\") pod \"kube-storage-version-migrator-operator-6c8f84f897-6wxf6\" (UID: \"3a305be2-8e7b-460c-adbf-691ae6c79683\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-6wxf6"
Sep 12 00:33:29.208463 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.208410    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc47c0e2-02fe-408b-b0a5-5b046a81f1d7-service-ca-bundle\") pod \"insights-operator-5844b856db-cnml8\" (UID: \"fc47c0e2-02fe-408b-b0a5-5b046a81f1d7\") " pod="openshift-insights/insights-operator-5844b856db-cnml8"
Sep 12 00:33:29.208573 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.208486    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dd90996a-5428-4ccf-a154-c0aa8def9789-trusted-ca\") pod \"console-operator-696884fb9c-czf8h\" (UID: \"dd90996a-5428-4ccf-a154-c0aa8def9789\") " pod="openshift-console-operator/console-operator-696884fb9c-czf8h"
Sep 12 00:33:29.208573 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.208547    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dd90996a-5428-4ccf-a154-c0aa8def9789-config\") pod \"console-operator-696884fb9c-czf8h\" (UID: \"dd90996a-5428-4ccf-a154-c0aa8def9789\") " pod="openshift-console-operator/console-operator-696884fb9c-czf8h"
Sep 12 00:33:29.208669 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.208580    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c47d77e0-78de-4d81-9e37-c9f5572e23a4-default-certificate\") pod \"router-default-d6d5fbc68-m67sb\" (UID: \"c47d77e0-78de-4d81-9e37-c9f5572e23a4\") " pod="openshift-ingress/router-default-d6d5fbc68-m67sb"
Sep 12 00:33:29.208669 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.208606    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c47d77e0-78de-4d81-9e37-c9f5572e23a4-service-ca-bundle\") pod \"router-default-d6d5fbc68-m67sb\" (UID: \"c47d77e0-78de-4d81-9e37-c9f5572e23a4\") " pod="openshift-ingress/router-default-d6d5fbc68-m67sb"
Sep 12 00:33:29.208669 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.208637    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/fc47c0e2-02fe-408b-b0a5-5b046a81f1d7-snapshots\") pod \"insights-operator-5844b856db-cnml8\" (UID: \"fc47c0e2-02fe-408b-b0a5-5b046a81f1d7\") " pod="openshift-insights/insights-operator-5844b856db-cnml8"
Sep 12 00:33:29.208669 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.208665    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc47c0e2-02fe-408b-b0a5-5b046a81f1d7-trusted-ca-bundle\") pod \"insights-operator-5844b856db-cnml8\" (UID: \"fc47c0e2-02fe-408b-b0a5-5b046a81f1d7\") " pod="openshift-insights/insights-operator-5844b856db-cnml8"
Sep 12 00:33:29.208857 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.208692    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fc47c0e2-02fe-408b-b0a5-5b046a81f1d7-serving-cert\") pod \"insights-operator-5844b856db-cnml8\" (UID: \"fc47c0e2-02fe-408b-b0a5-5b046a81f1d7\") " pod="openshift-insights/insights-operator-5844b856db-cnml8"
Sep 12 00:33:29.208857 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.208723    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4f2z\" (UniqueName: \"kubernetes.io/projected/c47d77e0-78de-4d81-9e37-c9f5572e23a4-kube-api-access-f4f2z\") pod \"router-default-d6d5fbc68-m67sb\" (UID: \"c47d77e0-78de-4d81-9e37-c9f5572e23a4\") " pod="openshift-ingress/router-default-d6d5fbc68-m67sb"
Sep 12 00:33:29.208857 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:29.208771    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c47d77e0-78de-4d81-9e37-c9f5572e23a4-service-ca-bundle podName:c47d77e0-78de-4d81-9e37-c9f5572e23a4 nodeName:}" failed. No retries permitted until 2026-09-12 00:33:29.708750001 +0000 UTC m=+246.382808858 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c47d77e0-78de-4d81-9e37-c9f5572e23a4-service-ca-bundle") pod "router-default-d6d5fbc68-m67sb" (UID: "c47d77e0-78de-4d81-9e37-c9f5572e23a4") : configmap references non-existent config key: service-ca.crt
Sep 12 00:33:29.208857 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.208801    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3a305be2-8e7b-460c-adbf-691ae6c79683-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-6wxf6\" (UID: \"3a305be2-8e7b-460c-adbf-691ae6c79683\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-6wxf6"
Sep 12 00:33:29.209149 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.208860    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dd90996a-5428-4ccf-a154-c0aa8def9789-serving-cert\") pod \"console-operator-696884fb9c-czf8h\" (UID: \"dd90996a-5428-4ccf-a154-c0aa8def9789\") " pod="openshift-console-operator/console-operator-696884fb9c-czf8h"
Sep 12 00:33:29.209149 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.208885    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssb9r\" (UniqueName: \"kubernetes.io/projected/dd90996a-5428-4ccf-a154-c0aa8def9789-kube-api-access-ssb9r\") pod \"console-operator-696884fb9c-czf8h\" (UID: \"dd90996a-5428-4ccf-a154-c0aa8def9789\") " pod="openshift-console-operator/console-operator-696884fb9c-czf8h"
Sep 12 00:33:29.209149 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.208930    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c47d77e0-78de-4d81-9e37-c9f5572e23a4-stats-auth\") pod \"router-default-d6d5fbc68-m67sb\" (UID: \"c47d77e0-78de-4d81-9e37-c9f5572e23a4\") " pod="openshift-ingress/router-default-d6d5fbc68-m67sb"
Sep 12 00:33:29.209149 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.209008    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc47c0e2-02fe-408b-b0a5-5b046a81f1d7-service-ca-bundle\") pod \"insights-operator-5844b856db-cnml8\" (UID: \"fc47c0e2-02fe-408b-b0a5-5b046a81f1d7\") " pod="openshift-insights/insights-operator-5844b856db-cnml8"
Sep 12 00:33:29.209149 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.209020    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c47d77e0-78de-4d81-9e37-c9f5572e23a4-metrics-certs\") pod \"router-default-d6d5fbc68-m67sb\" (UID: \"c47d77e0-78de-4d81-9e37-c9f5572e23a4\") " pod="openshift-ingress/router-default-d6d5fbc68-m67sb"
Sep 12 00:33:29.209149 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.209055    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3a305be2-8e7b-460c-adbf-691ae6c79683-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-6wxf6\" (UID: \"3a305be2-8e7b-460c-adbf-691ae6c79683\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-6wxf6"
Sep 12 00:33:29.209149 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:29.209109    2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 12 00:33:29.209149 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.209124    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc47c0e2-02fe-408b-b0a5-5b046a81f1d7-tmp\") pod \"insights-operator-5844b856db-cnml8\" (UID: \"fc47c0e2-02fe-408b-b0a5-5b046a81f1d7\") " pod="openshift-insights/insights-operator-5844b856db-cnml8"
Sep 12 00:33:29.209149 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:29.209153    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c47d77e0-78de-4d81-9e37-c9f5572e23a4-metrics-certs podName:c47d77e0-78de-4d81-9e37-c9f5572e23a4 nodeName:}" failed. No retries permitted until 2026-09-12 00:33:29.709136754 +0000 UTC m=+246.383195611 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c47d77e0-78de-4d81-9e37-c9f5572e23a4-metrics-certs") pod "router-default-d6d5fbc68-m67sb" (UID: "c47d77e0-78de-4d81-9e37-c9f5572e23a4") : secret "router-metrics-certs-default" not found
Sep 12 00:33:29.209597 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.209576    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/fc47c0e2-02fe-408b-b0a5-5b046a81f1d7-snapshots\") pod \"insights-operator-5844b856db-cnml8\" (UID: \"fc47c0e2-02fe-408b-b0a5-5b046a81f1d7\") " pod="openshift-insights/insights-operator-5844b856db-cnml8"
Sep 12 00:33:29.209661 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.209646    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc47c0e2-02fe-408b-b0a5-5b046a81f1d7-tmp\") pod \"insights-operator-5844b856db-cnml8\" (UID: \"fc47c0e2-02fe-408b-b0a5-5b046a81f1d7\") " pod="openshift-insights/insights-operator-5844b856db-cnml8"
Sep 12 00:33:29.209736 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.209721    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc47c0e2-02fe-408b-b0a5-5b046a81f1d7-trusted-ca-bundle\") pod \"insights-operator-5844b856db-cnml8\" (UID: \"fc47c0e2-02fe-408b-b0a5-5b046a81f1d7\") " pod="openshift-insights/insights-operator-5844b856db-cnml8"
Sep 12 00:33:29.211202 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.211183    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fc47c0e2-02fe-408b-b0a5-5b046a81f1d7-serving-cert\") pod \"insights-operator-5844b856db-cnml8\" (UID: \"fc47c0e2-02fe-408b-b0a5-5b046a81f1d7\") " pod="openshift-insights/insights-operator-5844b856db-cnml8"
Sep 12 00:33:29.211665 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.211649    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c47d77e0-78de-4d81-9e37-c9f5572e23a4-default-certificate\") pod \"router-default-d6d5fbc68-m67sb\" (UID: \"c47d77e0-78de-4d81-9e37-c9f5572e23a4\") " pod="openshift-ingress/router-default-d6d5fbc68-m67sb"
Sep 12 00:33:29.211789 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.211770    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c47d77e0-78de-4d81-9e37-c9f5572e23a4-stats-auth\") pod \"router-default-d6d5fbc68-m67sb\" (UID: \"c47d77e0-78de-4d81-9e37-c9f5572e23a4\") " pod="openshift-ingress/router-default-d6d5fbc68-m67sb"
Sep 12 00:33:29.218704 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.218678    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4djk\" (UniqueName: \"kubernetes.io/projected/fc47c0e2-02fe-408b-b0a5-5b046a81f1d7-kube-api-access-b4djk\") pod \"insights-operator-5844b856db-cnml8\" (UID: \"fc47c0e2-02fe-408b-b0a5-5b046a81f1d7\") " pod="openshift-insights/insights-operator-5844b856db-cnml8"
Sep 12 00:33:29.219099 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.219066    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4f2z\" (UniqueName: \"kubernetes.io/projected/c47d77e0-78de-4d81-9e37-c9f5572e23a4-kube-api-access-f4f2z\") pod \"router-default-d6d5fbc68-m67sb\" (UID: \"c47d77e0-78de-4d81-9e37-c9f5572e23a4\") " pod="openshift-ingress/router-default-d6d5fbc68-m67sb"
Sep 12 00:33:29.305187 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.305114    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-cnml8"
Sep 12 00:33:29.310276 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.310252    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3a305be2-8e7b-460c-adbf-691ae6c79683-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-6wxf6\" (UID: \"3a305be2-8e7b-460c-adbf-691ae6c79683\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-6wxf6"
Sep 12 00:33:29.310398 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.310295    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdjvb\" (UniqueName: \"kubernetes.io/projected/3a305be2-8e7b-460c-adbf-691ae6c79683-kube-api-access-jdjvb\") pod \"kube-storage-version-migrator-operator-6c8f84f897-6wxf6\" (UID: \"3a305be2-8e7b-460c-adbf-691ae6c79683\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-6wxf6"
Sep 12 00:33:29.310398 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.310314    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dd90996a-5428-4ccf-a154-c0aa8def9789-trusted-ca\") pod \"console-operator-696884fb9c-czf8h\" (UID: \"dd90996a-5428-4ccf-a154-c0aa8def9789\") " pod="openshift-console-operator/console-operator-696884fb9c-czf8h"
Sep 12 00:33:29.310398 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.310336    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dd90996a-5428-4ccf-a154-c0aa8def9789-config\") pod \"console-operator-696884fb9c-czf8h\" (UID: \"dd90996a-5428-4ccf-a154-c0aa8def9789\") " pod="openshift-console-operator/console-operator-696884fb9c-czf8h"
Sep 12 00:33:29.310398 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.310383    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3a305be2-8e7b-460c-adbf-691ae6c79683-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-6wxf6\" (UID: \"3a305be2-8e7b-460c-adbf-691ae6c79683\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-6wxf6"
Sep 12 00:33:29.310606 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.310428    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dd90996a-5428-4ccf-a154-c0aa8def9789-serving-cert\") pod \"console-operator-696884fb9c-czf8h\" (UID: \"dd90996a-5428-4ccf-a154-c0aa8def9789\") " pod="openshift-console-operator/console-operator-696884fb9c-czf8h"
Sep 12 00:33:29.310606 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.310449    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssb9r\" (UniqueName: \"kubernetes.io/projected/dd90996a-5428-4ccf-a154-c0aa8def9789-kube-api-access-ssb9r\") pod \"console-operator-696884fb9c-czf8h\" (UID: \"dd90996a-5428-4ccf-a154-c0aa8def9789\") " pod="openshift-console-operator/console-operator-696884fb9c-czf8h"
Sep 12 00:33:29.310933 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.310906    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3a305be2-8e7b-460c-adbf-691ae6c79683-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-6wxf6\" (UID: \"3a305be2-8e7b-460c-adbf-691ae6c79683\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-6wxf6"
Sep 12 00:33:29.311042 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.311010    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dd90996a-5428-4ccf-a154-c0aa8def9789-config\") pod \"console-operator-696884fb9c-czf8h\" (UID: \"dd90996a-5428-4ccf-a154-c0aa8def9789\") " pod="openshift-console-operator/console-operator-696884fb9c-czf8h"
Sep 12 00:33:29.312135 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.312112    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dd90996a-5428-4ccf-a154-c0aa8def9789-trusted-ca\") pod \"console-operator-696884fb9c-czf8h\" (UID: \"dd90996a-5428-4ccf-a154-c0aa8def9789\") " pod="openshift-console-operator/console-operator-696884fb9c-czf8h"
Sep 12 00:33:29.312890 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.312863    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dd90996a-5428-4ccf-a154-c0aa8def9789-serving-cert\") pod \"console-operator-696884fb9c-czf8h\" (UID: \"dd90996a-5428-4ccf-a154-c0aa8def9789\") " pod="openshift-console-operator/console-operator-696884fb9c-czf8h"
Sep 12 00:33:29.313050 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.313033    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3a305be2-8e7b-460c-adbf-691ae6c79683-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-6wxf6\" (UID: \"3a305be2-8e7b-460c-adbf-691ae6c79683\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-6wxf6"
Sep 12 00:33:29.317433 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.317410    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssb9r\" (UniqueName: \"kubernetes.io/projected/dd90996a-5428-4ccf-a154-c0aa8def9789-kube-api-access-ssb9r\") pod \"console-operator-696884fb9c-czf8h\" (UID: \"dd90996a-5428-4ccf-a154-c0aa8def9789\") " pod="openshift-console-operator/console-operator-696884fb9c-czf8h"
Sep 12 00:33:29.317706 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.317674    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdjvb\" (UniqueName: \"kubernetes.io/projected/3a305be2-8e7b-460c-adbf-691ae6c79683-kube-api-access-jdjvb\") pod \"kube-storage-version-migrator-operator-6c8f84f897-6wxf6\" (UID: \"3a305be2-8e7b-460c-adbf-691ae6c79683\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-6wxf6"
Sep 12 00:33:29.395464 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.395416    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-6wxf6"
Sep 12 00:33:29.401573 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.401549    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-czf8h"
Sep 12 00:33:29.571963 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.571888    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-cnml8"]
Sep 12 00:33:29.574926 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:33:29.574905    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc47c0e2_02fe_408b_b0a5_5b046a81f1d7.slice/crio-ef76847825a9e154b3cd3dd0df488198f04f17b074774eb0454774860af64bed WatchSource:0}: Error finding container ef76847825a9e154b3cd3dd0df488198f04f17b074774eb0454774860af64bed: Status 404 returned error can't find the container with id ef76847825a9e154b3cd3dd0df488198f04f17b074774eb0454774860af64bed
Sep 12 00:33:29.666315 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.666287    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-6wxf6"]
Sep 12 00:33:29.668898 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:33:29.668870    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a305be2_8e7b_460c_adbf_691ae6c79683.slice/crio-9d1389cf03dca9aafda318bc49948d37b8988be81fa94a8ae4b7005ae2bc5ae9 WatchSource:0}: Error finding container 9d1389cf03dca9aafda318bc49948d37b8988be81fa94a8ae4b7005ae2bc5ae9: Status 404 returned error can't find the container with id 9d1389cf03dca9aafda318bc49948d37b8988be81fa94a8ae4b7005ae2bc5ae9
Sep 12 00:33:29.672065 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.672045    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-czf8h"]
Sep 12 00:33:29.674912 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:33:29.674888    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd90996a_5428_4ccf_a154_c0aa8def9789.slice/crio-bd2e453cb0cfea245677b4a20b1c75db1802586e2e8af8ec5fb61a35849eb7f0 WatchSource:0}: Error finding container bd2e453cb0cfea245677b4a20b1c75db1802586e2e8af8ec5fb61a35849eb7f0: Status 404 returned error can't find the container with id bd2e453cb0cfea245677b4a20b1c75db1802586e2e8af8ec5fb61a35849eb7f0
Sep 12 00:33:29.713981 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.713947    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c47d77e0-78de-4d81-9e37-c9f5572e23a4-metrics-certs\") pod \"router-default-d6d5fbc68-m67sb\" (UID: \"c47d77e0-78de-4d81-9e37-c9f5572e23a4\") " pod="openshift-ingress/router-default-d6d5fbc68-m67sb"
Sep 12 00:33:29.714155 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:29.714013    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c47d77e0-78de-4d81-9e37-c9f5572e23a4-service-ca-bundle\") pod \"router-default-d6d5fbc68-m67sb\" (UID: \"c47d77e0-78de-4d81-9e37-c9f5572e23a4\") " pod="openshift-ingress/router-default-d6d5fbc68-m67sb"
Sep 12 00:33:29.714155 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:29.714102    2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 12 00:33:29.714155 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:29.714148    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c47d77e0-78de-4d81-9e37-c9f5572e23a4-service-ca-bundle podName:c47d77e0-78de-4d81-9e37-c9f5572e23a4 nodeName:}" failed. No retries permitted until 2026-09-12 00:33:30.714131283 +0000 UTC m=+247.388190121 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c47d77e0-78de-4d81-9e37-c9f5572e23a4-service-ca-bundle") pod "router-default-d6d5fbc68-m67sb" (UID: "c47d77e0-78de-4d81-9e37-c9f5572e23a4") : configmap references non-existent config key: service-ca.crt
Sep 12 00:33:29.714270 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:29.714163    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c47d77e0-78de-4d81-9e37-c9f5572e23a4-metrics-certs podName:c47d77e0-78de-4d81-9e37-c9f5572e23a4 nodeName:}" failed. No retries permitted until 2026-09-12 00:33:30.714157521 +0000 UTC m=+247.388216358 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c47d77e0-78de-4d81-9e37-c9f5572e23a4-metrics-certs") pod "router-default-d6d5fbc68-m67sb" (UID: "c47d77e0-78de-4d81-9e37-c9f5572e23a4") : secret "router-metrics-certs-default" not found
Sep 12 00:33:30.442473 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:30.442438    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-cnml8" event={"ID":"fc47c0e2-02fe-408b-b0a5-5b046a81f1d7","Type":"ContainerStarted","Data":"ef76847825a9e154b3cd3dd0df488198f04f17b074774eb0454774860af64bed"}
Sep 12 00:33:30.444176 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:30.444120    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-czf8h" event={"ID":"dd90996a-5428-4ccf-a154-c0aa8def9789","Type":"ContainerStarted","Data":"bd2e453cb0cfea245677b4a20b1c75db1802586e2e8af8ec5fb61a35849eb7f0"}
Sep 12 00:33:30.446848 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:30.446777    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-6wxf6" event={"ID":"3a305be2-8e7b-460c-adbf-691ae6c79683","Type":"ContainerStarted","Data":"9d1389cf03dca9aafda318bc49948d37b8988be81fa94a8ae4b7005ae2bc5ae9"}
Sep 12 00:33:30.724059 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:30.723970    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c47d77e0-78de-4d81-9e37-c9f5572e23a4-metrics-certs\") pod \"router-default-d6d5fbc68-m67sb\" (UID: \"c47d77e0-78de-4d81-9e37-c9f5572e23a4\") " pod="openshift-ingress/router-default-d6d5fbc68-m67sb"
Sep 12 00:33:30.724246 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:30.724102    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c47d77e0-78de-4d81-9e37-c9f5572e23a4-service-ca-bundle\") pod \"router-default-d6d5fbc68-m67sb\" (UID: \"c47d77e0-78de-4d81-9e37-c9f5572e23a4\") " pod="openshift-ingress/router-default-d6d5fbc68-m67sb"
Sep 12 00:33:30.724246 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:30.724158    2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 12 00:33:30.724362 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:30.724246    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c47d77e0-78de-4d81-9e37-c9f5572e23a4-service-ca-bundle podName:c47d77e0-78de-4d81-9e37-c9f5572e23a4 nodeName:}" failed. No retries permitted until 2026-09-12 00:33:32.724226232 +0000 UTC m=+249.398285091 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c47d77e0-78de-4d81-9e37-c9f5572e23a4-service-ca-bundle") pod "router-default-d6d5fbc68-m67sb" (UID: "c47d77e0-78de-4d81-9e37-c9f5572e23a4") : configmap references non-existent config key: service-ca.crt
Sep 12 00:33:30.724362 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:30.724272    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c47d77e0-78de-4d81-9e37-c9f5572e23a4-metrics-certs podName:c47d77e0-78de-4d81-9e37-c9f5572e23a4 nodeName:}" failed. No retries permitted until 2026-09-12 00:33:32.724261111 +0000 UTC m=+249.398319954 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c47d77e0-78de-4d81-9e37-c9f5572e23a4-metrics-certs") pod "router-default-d6d5fbc68-m67sb" (UID: "c47d77e0-78de-4d81-9e37-c9f5572e23a4") : secret "router-metrics-certs-default" not found
Sep 12 00:33:31.317439 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:31.317397    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:31.371320 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:31.371273    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:32.739630 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:32.739602    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c47d77e0-78de-4d81-9e37-c9f5572e23a4-metrics-certs\") pod \"router-default-d6d5fbc68-m67sb\" (UID: \"c47d77e0-78de-4d81-9e37-c9f5572e23a4\") " pod="openshift-ingress/router-default-d6d5fbc68-m67sb"
Sep 12 00:33:32.740012 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:32.739739    2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 12 00:33:32.740012 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:32.739812    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c47d77e0-78de-4d81-9e37-c9f5572e23a4-metrics-certs podName:c47d77e0-78de-4d81-9e37-c9f5572e23a4 nodeName:}" failed. No retries permitted until 2026-09-12 00:33:36.739793123 +0000 UTC m=+253.413851962 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c47d77e0-78de-4d81-9e37-c9f5572e23a4-metrics-certs") pod "router-default-d6d5fbc68-m67sb" (UID: "c47d77e0-78de-4d81-9e37-c9f5572e23a4") : secret "router-metrics-certs-default" not found
Sep 12 00:33:32.740012 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:32.739896    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c47d77e0-78de-4d81-9e37-c9f5572e23a4-service-ca-bundle\") pod \"router-default-d6d5fbc68-m67sb\" (UID: \"c47d77e0-78de-4d81-9e37-c9f5572e23a4\") " pod="openshift-ingress/router-default-d6d5fbc68-m67sb"
Sep 12 00:33:32.740012 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:32.739992    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c47d77e0-78de-4d81-9e37-c9f5572e23a4-service-ca-bundle podName:c47d77e0-78de-4d81-9e37-c9f5572e23a4 nodeName:}" failed. No retries permitted until 2026-09-12 00:33:36.739974301 +0000 UTC m=+253.414033145 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c47d77e0-78de-4d81-9e37-c9f5572e23a4-service-ca-bundle") pod "router-default-d6d5fbc68-m67sb" (UID: "c47d77e0-78de-4d81-9e37-c9f5572e23a4") : configmap references non-existent config key: service-ca.crt
Sep 12 00:33:32.900878 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:32.900800    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:32.947297 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:32.947266    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:33.140957 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:33.140924    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:33.189752 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:33.189722    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:33.457853 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:33.457770    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-6wxf6" event={"ID":"3a305be2-8e7b-460c-adbf-691ae6c79683","Type":"ContainerStarted","Data":"0a11e6ad6d329b5038b7292a3bf3eaffa7fc110ee132657b71b403ab70eabf6f"}
Sep 12 00:33:33.459368 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:33.459341    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-cnml8" event={"ID":"fc47c0e2-02fe-408b-b0a5-5b046a81f1d7","Type":"ContainerStarted","Data":"e8cc5985156bec469bc7d99aaa6346f8248554b5e90d9ac1a3210b97f798c925"}
Sep 12 00:33:33.460970 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:33.460949    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-czf8h_dd90996a-5428-4ccf-a154-c0aa8def9789/console-operator/0.log"
Sep 12 00:33:33.461079 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:33.460990    2558 generic.go:356] "Generic (PLEG): container finished" podID="dd90996a-5428-4ccf-a154-c0aa8def9789" containerID="e8fbf0916b7a716dee11d7b2f573ca6f633074bedc2451c1598787f93d0f29ad" exitCode=255
Sep 12 00:33:33.461079 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:33.461020    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-czf8h" event={"ID":"dd90996a-5428-4ccf-a154-c0aa8def9789","Type":"ContainerDied","Data":"e8fbf0916b7a716dee11d7b2f573ca6f633074bedc2451c1598787f93d0f29ad"}
Sep 12 00:33:33.461274 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:33.461258    2558 scope.go:117] "RemoveContainer" containerID="e8fbf0916b7a716dee11d7b2f573ca6f633074bedc2451c1598787f93d0f29ad"
Sep 12 00:33:33.471811 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:33.471768    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-6wxf6" podStartSLOduration=1.54870594 podStartE2EDuration="4.471751538s" totalImagesPullingTime="2.923045598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:33:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:33:33.471566387 +0000 UTC m=+250.145625231" watchObservedRunningTime="2026-09-12 00:33:33.471751538 +0000 UTC m=+250.145810398"
Sep 12 00:33:33.504871 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:33.504829    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-cnml8" podStartSLOduration=2.493022881 podStartE2EDuration="5.504812237s" totalImagesPullingTime="3.011789356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:33:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:33:33.503899628 +0000 UTC m=+250.177958511" watchObservedRunningTime="2026-09-12 00:33:33.504812237 +0000 UTC m=+250.178871096"
Sep 12 00:33:33.611759 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:33.611730    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:33.712236 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:33.712157    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:33.790757 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:33.790723    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:33.849735 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:33.849699    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:33.953628 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:33.953599    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:34.465168 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:34.465143    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-czf8h_dd90996a-5428-4ccf-a154-c0aa8def9789/console-operator/1.log"
Sep 12 00:33:34.465558 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:34.465544    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-czf8h_dd90996a-5428-4ccf-a154-c0aa8def9789/console-operator/0.log"
Sep 12 00:33:34.465601 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:34.465581    2558 generic.go:356] "Generic (PLEG): container finished" podID="dd90996a-5428-4ccf-a154-c0aa8def9789" containerID="698a8f12f3b562ac66a9da800786fb3302e3d5d802c877d50ca79095b20bc62a" exitCode=255
Sep 12 00:33:34.465670 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:34.465654    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-czf8h" event={"ID":"dd90996a-5428-4ccf-a154-c0aa8def9789","Type":"ContainerDied","Data":"698a8f12f3b562ac66a9da800786fb3302e3d5d802c877d50ca79095b20bc62a"}
Sep 12 00:33:34.465710 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:34.465702    2558 scope.go:117] "RemoveContainer" containerID="e8fbf0916b7a716dee11d7b2f573ca6f633074bedc2451c1598787f93d0f29ad"
Sep 12 00:33:34.465895 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:34.465853    2558 scope.go:117] "RemoveContainer" containerID="698a8f12f3b562ac66a9da800786fb3302e3d5d802c877d50ca79095b20bc62a"
Sep 12 00:33:34.466066 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:34.466043    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-czf8h_openshift-console-operator(dd90996a-5428-4ccf-a154-c0aa8def9789)\"" pod="openshift-console-operator/console-operator-696884fb9c-czf8h" podUID="dd90996a-5428-4ccf-a154-c0aa8def9789"
Sep 12 00:33:34.634783 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:34.634747    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-s4wrn"]
Sep 12 00:33:34.639199 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:34.639178    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-s4wrn"
Sep 12 00:33:34.642108 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:34.642072    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-hwlcm\""
Sep 12 00:33:34.642473 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:34.642447    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 12 00:33:34.642780 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:34.642759    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 12 00:33:34.654330 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:34.654309    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/97666338-a119-4951-baeb-f794d6e6ac0b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s4wrn\" (UID: \"97666338-a119-4951-baeb-f794d6e6ac0b\") " pod="openshift-insights/insights-runtime-extractor-s4wrn"
Sep 12 00:33:34.654437 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:34.654339    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwcd4\" (UniqueName: \"kubernetes.io/projected/97666338-a119-4951-baeb-f794d6e6ac0b-kube-api-access-jwcd4\") pod \"insights-runtime-extractor-s4wrn\" (UID: \"97666338-a119-4951-baeb-f794d6e6ac0b\") " pod="openshift-insights/insights-runtime-extractor-s4wrn"
Sep 12 00:33:34.654437 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:34.654386    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/97666338-a119-4951-baeb-f794d6e6ac0b-data-volume\") pod \"insights-runtime-extractor-s4wrn\" (UID: \"97666338-a119-4951-baeb-f794d6e6ac0b\") " pod="openshift-insights/insights-runtime-extractor-s4wrn"
Sep 12 00:33:34.654437 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:34.654416    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/97666338-a119-4951-baeb-f794d6e6ac0b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s4wrn\" (UID: \"97666338-a119-4951-baeb-f794d6e6ac0b\") " pod="openshift-insights/insights-runtime-extractor-s4wrn"
Sep 12 00:33:34.654437 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:34.654435    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/97666338-a119-4951-baeb-f794d6e6ac0b-crio-socket\") pod \"insights-runtime-extractor-s4wrn\" (UID: \"97666338-a119-4951-baeb-f794d6e6ac0b\") " pod="openshift-insights/insights-runtime-extractor-s4wrn"
Sep 12 00:33:34.661594 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:34.661574    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:34.664883 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:34.664862    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-s4wrn"]
Sep 12 00:33:34.737291 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:34.737218    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:34.755016 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:34.754985    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/97666338-a119-4951-baeb-f794d6e6ac0b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s4wrn\" (UID: \"97666338-a119-4951-baeb-f794d6e6ac0b\") " pod="openshift-insights/insights-runtime-extractor-s4wrn"
Sep 12 00:33:34.755200 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:34.755023    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwcd4\" (UniqueName: \"kubernetes.io/projected/97666338-a119-4951-baeb-f794d6e6ac0b-kube-api-access-jwcd4\") pod \"insights-runtime-extractor-s4wrn\" (UID: \"97666338-a119-4951-baeb-f794d6e6ac0b\") " pod="openshift-insights/insights-runtime-extractor-s4wrn"
Sep 12 00:33:34.755200 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:34.755105    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/97666338-a119-4951-baeb-f794d6e6ac0b-data-volume\") pod \"insights-runtime-extractor-s4wrn\" (UID: \"97666338-a119-4951-baeb-f794d6e6ac0b\") " pod="openshift-insights/insights-runtime-extractor-s4wrn"
Sep 12 00:33:34.755200 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:34.755139    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 12 00:33:34.755200 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:34.755156    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/97666338-a119-4951-baeb-f794d6e6ac0b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s4wrn\" (UID: \"97666338-a119-4951-baeb-f794d6e6ac0b\") " pod="openshift-insights/insights-runtime-extractor-s4wrn"
Sep 12 00:33:34.755200 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:34.755182    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/97666338-a119-4951-baeb-f794d6e6ac0b-crio-socket\") pod \"insights-runtime-extractor-s4wrn\" (UID: \"97666338-a119-4951-baeb-f794d6e6ac0b\") " pod="openshift-insights/insights-runtime-extractor-s4wrn"
Sep 12 00:33:34.755200 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:34.755200    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97666338-a119-4951-baeb-f794d6e6ac0b-insights-runtime-extractor-tls podName:97666338-a119-4951-baeb-f794d6e6ac0b nodeName:}" failed. No retries permitted until 2026-09-12 00:33:35.255186139 +0000 UTC m=+251.929244980 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/97666338-a119-4951-baeb-f794d6e6ac0b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s4wrn" (UID: "97666338-a119-4951-baeb-f794d6e6ac0b") : secret "insights-runtime-extractor-tls" not found
Sep 12 00:33:34.755461 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:34.755237    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/97666338-a119-4951-baeb-f794d6e6ac0b-crio-socket\") pod \"insights-runtime-extractor-s4wrn\" (UID: \"97666338-a119-4951-baeb-f794d6e6ac0b\") " pod="openshift-insights/insights-runtime-extractor-s4wrn"
Sep 12 00:33:34.755461 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:34.755275    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 12 00:33:34.755461 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:34.755344    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/97666338-a119-4951-baeb-f794d6e6ac0b-kube-rbac-proxy-cm podName:97666338-a119-4951-baeb-f794d6e6ac0b nodeName:}" failed. No retries permitted until 2026-09-12 00:33:35.255328264 +0000 UTC m=+251.929387115 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/97666338-a119-4951-baeb-f794d6e6ac0b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s4wrn" (UID: "97666338-a119-4951-baeb-f794d6e6ac0b") : configmap "kube-rbac-proxy" not found
Sep 12 00:33:34.755565 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:34.755495    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/97666338-a119-4951-baeb-f794d6e6ac0b-data-volume\") pod \"insights-runtime-extractor-s4wrn\" (UID: \"97666338-a119-4951-baeb-f794d6e6ac0b\") " pod="openshift-insights/insights-runtime-extractor-s4wrn"
Sep 12 00:33:34.763231 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:34.763213    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwcd4\" (UniqueName: \"kubernetes.io/projected/97666338-a119-4951-baeb-f794d6e6ac0b-kube-api-access-jwcd4\") pod \"insights-runtime-extractor-s4wrn\" (UID: \"97666338-a119-4951-baeb-f794d6e6ac0b\") " pod="openshift-insights/insights-runtime-extractor-s4wrn"
Sep 12 00:33:35.257940 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:35.257901    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/97666338-a119-4951-baeb-f794d6e6ac0b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s4wrn\" (UID: \"97666338-a119-4951-baeb-f794d6e6ac0b\") " pod="openshift-insights/insights-runtime-extractor-s4wrn"
Sep 12 00:33:35.258330 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:35.257962    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/97666338-a119-4951-baeb-f794d6e6ac0b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s4wrn\" (UID: \"97666338-a119-4951-baeb-f794d6e6ac0b\") " pod="openshift-insights/insights-runtime-extractor-s4wrn"
Sep 12 00:33:35.258330 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:35.258019    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 12 00:33:35.258330 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:35.258062    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 12 00:33:35.258330 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:35.258117    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/97666338-a119-4951-baeb-f794d6e6ac0b-kube-rbac-proxy-cm podName:97666338-a119-4951-baeb-f794d6e6ac0b nodeName:}" failed. No retries permitted until 2026-09-12 00:33:36.258080919 +0000 UTC m=+252.932139756 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/97666338-a119-4951-baeb-f794d6e6ac0b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s4wrn" (UID: "97666338-a119-4951-baeb-f794d6e6ac0b") : configmap "kube-rbac-proxy" not found
Sep 12 00:33:35.258330 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:35.258133    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97666338-a119-4951-baeb-f794d6e6ac0b-insights-runtime-extractor-tls podName:97666338-a119-4951-baeb-f794d6e6ac0b nodeName:}" failed. No retries permitted until 2026-09-12 00:33:36.258127133 +0000 UTC m=+252.932185970 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/97666338-a119-4951-baeb-f794d6e6ac0b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s4wrn" (UID: "97666338-a119-4951-baeb-f794d6e6ac0b") : secret "insights-runtime-extractor-tls" not found
Sep 12 00:33:35.313137 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:35.313104    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:35.360443 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:35.360412    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:35.472959 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:35.472934    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-czf8h_dd90996a-5428-4ccf-a154-c0aa8def9789/console-operator/1.log"
Sep 12 00:33:35.473286 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:35.473272    2558 scope.go:117] "RemoveContainer" containerID="698a8f12f3b562ac66a9da800786fb3302e3d5d802c877d50ca79095b20bc62a"
Sep 12 00:33:35.473445 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:35.473428    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-czf8h_openshift-console-operator(dd90996a-5428-4ccf-a154-c0aa8def9789)\"" pod="openshift-console-operator/console-operator-696884fb9c-czf8h" podUID="dd90996a-5428-4ccf-a154-c0aa8def9789"
Sep 12 00:33:35.659779 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:35.659681    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/746054e5-684e-4c55-824b-6fdde562eb02-metrics-certs\") pod \"network-metrics-daemon-7tkvx\" (UID: \"746054e5-684e-4c55-824b-6fdde562eb02\") " pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:33:35.659938 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:35.659803    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 12 00:33:35.659938 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:35.659853    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/746054e5-684e-4c55-824b-6fdde562eb02-metrics-certs podName:746054e5-684e-4c55-824b-6fdde562eb02 nodeName:}" failed. No retries permitted until 2026-09-12 00:35:37.659840267 +0000 UTC m=+374.333899104 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/746054e5-684e-4c55-824b-6fdde562eb02-metrics-certs") pod "network-metrics-daemon-7tkvx" (UID: "746054e5-684e-4c55-824b-6fdde562eb02") : secret "metrics-daemon-secret" not found
Sep 12 00:33:36.059509 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:36.059482    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:36.265166 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:36.265131    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/97666338-a119-4951-baeb-f794d6e6ac0b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s4wrn\" (UID: \"97666338-a119-4951-baeb-f794d6e6ac0b\") " pod="openshift-insights/insights-runtime-extractor-s4wrn"
Sep 12 00:33:36.265528 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:36.265248    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/97666338-a119-4951-baeb-f794d6e6ac0b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s4wrn\" (UID: \"97666338-a119-4951-baeb-f794d6e6ac0b\") " pod="openshift-insights/insights-runtime-extractor-s4wrn"
Sep 12 00:33:36.265528 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:36.265295    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 12 00:33:36.265528 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:36.265361    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/97666338-a119-4951-baeb-f794d6e6ac0b-kube-rbac-proxy-cm podName:97666338-a119-4951-baeb-f794d6e6ac0b nodeName:}" failed. No retries permitted until 2026-09-12 00:33:38.265342562 +0000 UTC m=+254.939401419 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/97666338-a119-4951-baeb-f794d6e6ac0b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s4wrn" (UID: "97666338-a119-4951-baeb-f794d6e6ac0b") : configmap "kube-rbac-proxy" not found
Sep 12 00:33:36.265528 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:36.265362    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 12 00:33:36.265528 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:36.265397    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97666338-a119-4951-baeb-f794d6e6ac0b-insights-runtime-extractor-tls podName:97666338-a119-4951-baeb-f794d6e6ac0b nodeName:}" failed. No retries permitted until 2026-09-12 00:33:38.265387416 +0000 UTC m=+254.939446253 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/97666338-a119-4951-baeb-f794d6e6ac0b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s4wrn" (UID: "97666338-a119-4951-baeb-f794d6e6ac0b") : secret "insights-runtime-extractor-tls" not found
Sep 12 00:33:36.768252 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:36.768207    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c47d77e0-78de-4d81-9e37-c9f5572e23a4-metrics-certs\") pod \"router-default-d6d5fbc68-m67sb\" (UID: \"c47d77e0-78de-4d81-9e37-c9f5572e23a4\") " pod="openshift-ingress/router-default-d6d5fbc68-m67sb"
Sep 12 00:33:36.768434 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:36.768291    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c47d77e0-78de-4d81-9e37-c9f5572e23a4-service-ca-bundle\") pod \"router-default-d6d5fbc68-m67sb\" (UID: \"c47d77e0-78de-4d81-9e37-c9f5572e23a4\") " pod="openshift-ingress/router-default-d6d5fbc68-m67sb"
Sep 12 00:33:36.768434 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:36.768356    2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 12 00:33:36.768434 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:36.768419    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c47d77e0-78de-4d81-9e37-c9f5572e23a4-service-ca-bundle podName:c47d77e0-78de-4d81-9e37-c9f5572e23a4 nodeName:}" failed. No retries permitted until 2026-09-12 00:33:44.768405171 +0000 UTC m=+261.442464008 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c47d77e0-78de-4d81-9e37-c9f5572e23a4-service-ca-bundle") pod "router-default-d6d5fbc68-m67sb" (UID: "c47d77e0-78de-4d81-9e37-c9f5572e23a4") : configmap references non-existent config key: service-ca.crt
Sep 12 00:33:36.768434 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:36.768434    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c47d77e0-78de-4d81-9e37-c9f5572e23a4-metrics-certs podName:c47d77e0-78de-4d81-9e37-c9f5572e23a4 nodeName:}" failed. No retries permitted until 2026-09-12 00:33:44.768427422 +0000 UTC m=+261.442486259 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c47d77e0-78de-4d81-9e37-c9f5572e23a4-metrics-certs") pod "router-default-d6d5fbc68-m67sb" (UID: "c47d77e0-78de-4d81-9e37-c9f5572e23a4") : secret "router-metrics-certs-default" not found
Sep 12 00:33:36.886698 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:36.886665    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:37.020471 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:37.020397    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:37.074786 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:37.074759    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:37.208395 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:37.208364    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gs67t_34aa479e-166d-4d92-8909-a58f6e2649ef/dns-node-resolver/0.log"
Sep 12 00:33:37.293201 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:37.293123    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:37.393529 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:37.393494    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:37.589314 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:37.589233    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:38.280153 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:38.280115    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/97666338-a119-4951-baeb-f794d6e6ac0b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s4wrn\" (UID: \"97666338-a119-4951-baeb-f794d6e6ac0b\") " pod="openshift-insights/insights-runtime-extractor-s4wrn"
Sep 12 00:33:38.280341 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:38.280215    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/97666338-a119-4951-baeb-f794d6e6ac0b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s4wrn\" (UID: \"97666338-a119-4951-baeb-f794d6e6ac0b\") " pod="openshift-insights/insights-runtime-extractor-s4wrn"
Sep 12 00:33:38.280341 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:38.280235    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 12 00:33:38.280341 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:38.280296    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/97666338-a119-4951-baeb-f794d6e6ac0b-kube-rbac-proxy-cm podName:97666338-a119-4951-baeb-f794d6e6ac0b nodeName:}" failed. No retries permitted until 2026-09-12 00:33:42.280278858 +0000 UTC m=+258.954337729 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/97666338-a119-4951-baeb-f794d6e6ac0b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s4wrn" (UID: "97666338-a119-4951-baeb-f794d6e6ac0b") : configmap "kube-rbac-proxy" not found
Sep 12 00:33:38.280443 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:38.280362    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 12 00:33:38.280443 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:38.280429    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97666338-a119-4951-baeb-f794d6e6ac0b-insights-runtime-extractor-tls podName:97666338-a119-4951-baeb-f794d6e6ac0b nodeName:}" failed. No retries permitted until 2026-09-12 00:33:42.280412416 +0000 UTC m=+258.954471258 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/97666338-a119-4951-baeb-f794d6e6ac0b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s4wrn" (UID: "97666338-a119-4951-baeb-f794d6e6ac0b") : secret "insights-runtime-extractor-tls" not found
Sep 12 00:33:38.389067 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:38.389032    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:38.408126 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:38.408109    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fd27q_c17577fa-fd1b-45a2-9d4d-6be7144ba759/node-ca/0.log"
Sep 12 00:33:38.928393 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:38.928359    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:39.029534 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:39.029506    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:39.194494 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:39.194458    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:39.402067 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:39.402033    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-czf8h"
Sep 12 00:33:39.402486 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:39.402107    2558 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-czf8h"
Sep 12 00:33:39.402486 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:39.402427    2558 scope.go:117] "RemoveContainer" containerID="698a8f12f3b562ac66a9da800786fb3302e3d5d802c877d50ca79095b20bc62a"
Sep 12 00:33:39.402595 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:39.402577    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-czf8h_openshift-console-operator(dd90996a-5428-4ccf-a154-c0aa8def9789)\"" pod="openshift-console-operator/console-operator-696884fb9c-czf8h" podUID="dd90996a-5428-4ccf-a154-c0aa8def9789"
Sep 12 00:33:39.408695 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:39.408675    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vcclb_421f63c6-8abc-4091-a546-2754a4f2066f/serve-healthcheck-canary/0.log"
Sep 12 00:33:39.588483 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:39.588372    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:39.785600 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:39.785569    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:40.009029 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:40.009000    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-6wxf6_3a305be2-8e7b-460c-adbf-691ae6c79683/kube-storage-version-migrator-operator/0.log"
Sep 12 00:33:40.389664 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:40.389587    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:40.588756 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:40.588718    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:40.789580 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:40.789541    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:40.983648 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:40.983619    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:41.383525 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:41.383495    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:41.587067 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:41.587032    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:42.185779 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:42.185746    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:42.239287 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:42.239254    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:42.313912 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:42.313889    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/97666338-a119-4951-baeb-f794d6e6ac0b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s4wrn\" (UID: \"97666338-a119-4951-baeb-f794d6e6ac0b\") " pod="openshift-insights/insights-runtime-extractor-s4wrn"
Sep 12 00:33:42.314020 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:42.313945    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/97666338-a119-4951-baeb-f794d6e6ac0b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s4wrn\" (UID: \"97666338-a119-4951-baeb-f794d6e6ac0b\") " pod="openshift-insights/insights-runtime-extractor-s4wrn"
Sep 12 00:33:42.314020 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:42.314002    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 12 00:33:42.314110 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:42.314062    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/97666338-a119-4951-baeb-f794d6e6ac0b-kube-rbac-proxy-cm podName:97666338-a119-4951-baeb-f794d6e6ac0b nodeName:}" failed. No retries permitted until 2026-09-12 00:33:50.314048111 +0000 UTC m=+266.988106947 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/97666338-a119-4951-baeb-f794d6e6ac0b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s4wrn" (UID: "97666338-a119-4951-baeb-f794d6e6ac0b") : configmap "kube-rbac-proxy" not found
Sep 12 00:33:42.314110 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:42.314066    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 12 00:33:42.314223 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:42.314125    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97666338-a119-4951-baeb-f794d6e6ac0b-insights-runtime-extractor-tls podName:97666338-a119-4951-baeb-f794d6e6ac0b nodeName:}" failed. No retries permitted until 2026-09-12 00:33:50.314111675 +0000 UTC m=+266.988170511 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/97666338-a119-4951-baeb-f794d6e6ac0b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s4wrn" (UID: "97666338-a119-4951-baeb-f794d6e6ac0b") : secret "insights-runtime-extractor-tls" not found
Sep 12 00:33:42.385290 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:42.385254    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:42.589334 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:42.589261    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:42.987945 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:42.987915    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:43.575764 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:43.575731    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:43.677504 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:43.677473    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:44.188810 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:44.188771    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:44.417017 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:44.416981    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:44.583055 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:44.582969    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:44.833556 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:44.833476    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c47d77e0-78de-4d81-9e37-c9f5572e23a4-metrics-certs\") pod \"router-default-d6d5fbc68-m67sb\" (UID: \"c47d77e0-78de-4d81-9e37-c9f5572e23a4\") " pod="openshift-ingress/router-default-d6d5fbc68-m67sb"
Sep 12 00:33:44.833556 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:44.833542    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c47d77e0-78de-4d81-9e37-c9f5572e23a4-service-ca-bundle\") pod \"router-default-d6d5fbc68-m67sb\" (UID: \"c47d77e0-78de-4d81-9e37-c9f5572e23a4\") " pod="openshift-ingress/router-default-d6d5fbc68-m67sb"
Sep 12 00:33:44.834059 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:44.834043    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c47d77e0-78de-4d81-9e37-c9f5572e23a4-service-ca-bundle\") pod \"router-default-d6d5fbc68-m67sb\" (UID: \"c47d77e0-78de-4d81-9e37-c9f5572e23a4\") " pod="openshift-ingress/router-default-d6d5fbc68-m67sb"
Sep 12 00:33:44.835929 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:44.835902    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c47d77e0-78de-4d81-9e37-c9f5572e23a4-metrics-certs\") pod \"router-default-d6d5fbc68-m67sb\" (UID: \"c47d77e0-78de-4d81-9e37-c9f5572e23a4\") " pod="openshift-ingress/router-default-d6d5fbc68-m67sb"
Sep 12 00:33:44.899281 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:44.899242    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-d6d5fbc68-m67sb"
Sep 12 00:33:44.984187 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:44.984158    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:45.173264 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:45.173241    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-d6d5fbc68-m67sb"]
Sep 12 00:33:45.175600 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:33:45.175574    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc47d77e0_78de_4d81_9e37_c9f5572e23a4.slice/crio-236fb43dfce1e6f997697e87f37796203398a5b4d6d26b538847f4b6e4ec992e WatchSource:0}: Error finding container 236fb43dfce1e6f997697e87f37796203398a5b4d6d26b538847f4b6e4ec992e: Status 404 returned error can't find the container with id 236fb43dfce1e6f997697e87f37796203398a5b4d6d26b538847f4b6e4ec992e
Sep 12 00:33:45.388128 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:45.388017    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:45.505646 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:45.505617    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-d6d5fbc68-m67sb" event={"ID":"c47d77e0-78de-4d81-9e37-c9f5572e23a4","Type":"ContainerStarted","Data":"545b1d1cd24c80e4f2d539c60a85937934d668319345b182bb3a6cae1fa4a88c"}
Sep 12 00:33:45.505814 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:45.505659    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-d6d5fbc68-m67sb" event={"ID":"c47d77e0-78de-4d81-9e37-c9f5572e23a4","Type":"ContainerStarted","Data":"236fb43dfce1e6f997697e87f37796203398a5b4d6d26b538847f4b6e4ec992e"}
Sep 12 00:33:45.521869 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:45.521820    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-d6d5fbc68-m67sb" podStartSLOduration=17.521799621 podStartE2EDuration="17.521799621s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:33:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:33:45.521644393 +0000 UTC m=+262.195703252" watchObservedRunningTime="2026-09-12 00:33:45.521799621 +0000 UTC m=+262.195858481"
Sep 12 00:33:45.588514 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:45.588482    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:45.899986 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:45.899903    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-d6d5fbc68-m67sb"
Sep 12 00:33:45.902528 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:45.902501    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-d6d5fbc68-m67sb"
Sep 12 00:33:46.316060 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:46.315996    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:46.415876 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:46.415844    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:46.509677 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:46.509639    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-d6d5fbc68-m67sb"
Sep 12 00:33:46.510836 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:46.510819    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-d6d5fbc68-m67sb"
Sep 12 00:33:46.582375 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:46.582295    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:46.987298 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:46.987268    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:47.185428 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:47.185396    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:47.536059 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:47.536032    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:47.638886 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:47.638849    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:47.736776 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:47.736747    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:47.984246 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:47.984220    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:50.380457 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:50.380420    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/97666338-a119-4951-baeb-f794d6e6ac0b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s4wrn\" (UID: \"97666338-a119-4951-baeb-f794d6e6ac0b\") " pod="openshift-insights/insights-runtime-extractor-s4wrn"
Sep 12 00:33:50.380904 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:50.380476    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/97666338-a119-4951-baeb-f794d6e6ac0b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s4wrn\" (UID: \"97666338-a119-4951-baeb-f794d6e6ac0b\") " pod="openshift-insights/insights-runtime-extractor-s4wrn"
Sep 12 00:33:50.380982 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:50.380962    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/97666338-a119-4951-baeb-f794d6e6ac0b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s4wrn\" (UID: \"97666338-a119-4951-baeb-f794d6e6ac0b\") " pod="openshift-insights/insights-runtime-extractor-s4wrn"
Sep 12 00:33:50.382965 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:50.382942    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/97666338-a119-4951-baeb-f794d6e6ac0b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s4wrn\" (UID: \"97666338-a119-4951-baeb-f794d6e6ac0b\") " pod="openshift-insights/insights-runtime-extractor-s4wrn"
Sep 12 00:33:50.548291 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:50.548256    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-s4wrn"
Sep 12 00:33:50.816619 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:50.816593    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-s4wrn"]
Sep 12 00:33:50.818871 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:33:50.818848    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97666338_a119_4951_baeb_f794d6e6ac0b.slice/crio-ddb74d8643225050c7427d330bddb414cef1bf08431bd87afa2fd17847ce8c81 WatchSource:0}: Error finding container ddb74d8643225050c7427d330bddb414cef1bf08431bd87afa2fd17847ce8c81: Status 404 returned error can't find the container with id ddb74d8643225050c7427d330bddb414cef1bf08431bd87afa2fd17847ce8c81
Sep 12 00:33:51.528999 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:51.528961    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s4wrn" event={"ID":"97666338-a119-4951-baeb-f794d6e6ac0b","Type":"ContainerStarted","Data":"1cf3a6a613bd15f1ca4977fa9f41e1fe1c7c520910aad9d21973fd1ed4fe9a45"}
Sep 12 00:33:51.529389 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:51.529014    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s4wrn" event={"ID":"97666338-a119-4951-baeb-f794d6e6ac0b","Type":"ContainerStarted","Data":"ddb74d8643225050c7427d330bddb414cef1bf08431bd87afa2fd17847ce8c81"}
Sep 12 00:33:52.534658 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:52.534623    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s4wrn" event={"ID":"97666338-a119-4951-baeb-f794d6e6ac0b","Type":"ContainerStarted","Data":"da2b8e8fbb15b5729ec6410fa782201c6c61f5300f77181fb59de7ebe5f45b60"}
Sep 12 00:33:53.543622 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:53.543579    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s4wrn" event={"ID":"97666338-a119-4951-baeb-f794d6e6ac0b","Type":"ContainerStarted","Data":"86f6beb0723f6aaf31b4827bacf7350e69e1585410a520a843caf0374b1a21f7"}
Sep 12 00:33:53.560104 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:53.560041    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-s4wrn" podStartSLOduration=17.588986975 podStartE2EDuration="19.560025026s" totalImagesPullingTime="1.971038051s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 00:33:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:33:53.558619064 +0000 UTC m=+270.232677936" watchObservedRunningTime="2026-09-12 00:33:53.560025026 +0000 UTC m=+270.234083885"
Sep 12 00:33:53.733671 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:53.733638    2558 scope.go:117] "RemoveContainer" containerID="698a8f12f3b562ac66a9da800786fb3302e3d5d802c877d50ca79095b20bc62a"
Sep 12 00:33:54.548633 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:54.548606    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-czf8h_dd90996a-5428-4ccf-a154-c0aa8def9789/console-operator/1.log"
Sep 12 00:33:54.549027 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:54.548722    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-czf8h" event={"ID":"dd90996a-5428-4ccf-a154-c0aa8def9789","Type":"ContainerStarted","Data":"37470c4c5d333dfa866fd730c72a02dbe31a74bc78959143115e2a636cace17c"}
Sep 12 00:33:54.549139 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:54.549119    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-czf8h"
Sep 12 00:33:54.564988 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:54.564941    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-czf8h" podStartSLOduration=22.648412525 podStartE2EDuration="25.564922956s" totalImagesPullingTime="2.916510431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:33:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:33:54.564520156 +0000 UTC m=+271.238579012" watchObservedRunningTime="2026-09-12 00:33:54.564922956 +0000 UTC m=+271.238981819"
Sep 12 00:33:55.196477 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:55.196435    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:55.251796 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:55.251757    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:55.549820 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:55.549719    2558 patch_prober.go:28] interesting pod/console-operator-696884fb9c-czf8h container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.133.0.12:8443/readyz\": context deadline exceeded" start-of-body=
Sep 12 00:33:55.549820 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:55.549782    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-696884fb9c-czf8h" podUID="dd90996a-5428-4ccf-a154-c0aa8def9789" containerName="console-operator" probeResult="failure" output="Get \"https://10.133.0.12:8443/readyz\": context deadline exceeded"
Sep 12 00:33:55.727501 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:55.727470    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-czf8h"
Sep 12 00:33:55.909512 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:55.909437    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:56.006380 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:56.006346    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:56.205587 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:56.205560    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:56.578761 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:56.578676    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:56.629506 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:56.629472    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:56.983367 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:56.983326    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:57.034410 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:57.034378    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:57.180322 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:57.180285    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:57.231886 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:57.231855    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:57.814861 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:57.814828    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-55c4s"]
Sep 12 00:33:57.819005 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:57.818980    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-55c4s"
Sep 12 00:33:57.828164 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:57.828138    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 12 00:33:57.828332 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:57.828138    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 12 00:33:57.830543 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:57.830512    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-599f5d46cb-zk8ln"]
Sep 12 00:33:57.830794 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:33:57.830769    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln" podUID="dbe5b08b-127a-4b6c-90db-7abd802f6e9a"
Sep 12 00:33:57.831271 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:57.831253    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-7zcbg\""
Sep 12 00:33:57.850983 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:57.850950    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-55c4s"]
Sep 12 00:33:57.880446 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:57.880410    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:57.944635 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:57.944600    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bl8q\" (UniqueName: \"kubernetes.io/projected/df843273-bc32-44d4-95bd-0ed4b1db206c-kube-api-access-9bl8q\") pod \"downloads-76f99584f4-55c4s\" (UID: \"df843273-bc32-44d4-95bd-0ed4b1db206c\") " pod="openshift-console/downloads-76f99584f4-55c4s"
Sep 12 00:33:57.972256 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:57.972221    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:58.042533 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:58.042499    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:58.045157 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:58.045138    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bl8q\" (UniqueName: \"kubernetes.io/projected/df843273-bc32-44d4-95bd-0ed4b1db206c-kube-api-access-9bl8q\") pod \"downloads-76f99584f4-55c4s\" (UID: \"df843273-bc32-44d4-95bd-0ed4b1db206c\") " pod="openshift-console/downloads-76f99584f4-55c4s"
Sep 12 00:33:58.056114 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:58.056059    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bl8q\" (UniqueName: \"kubernetes.io/projected/df843273-bc32-44d4-95bd-0ed4b1db206c-kube-api-access-9bl8q\") pod \"downloads-76f99584f4-55c4s\" (UID: \"df843273-bc32-44d4-95bd-0ed4b1db206c\") " pod="openshift-console/downloads-76f99584f4-55c4s"
Sep 12 00:33:58.128894 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:58.128817    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-55c4s"
Sep 12 00:33:58.397537 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:58.397463    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-55c4s"]
Sep 12 00:33:58.400033 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:33:58.400005    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf843273_bc32_44d4_95bd_0ed4b1db206c.slice/crio-1eac5295d6cb6eb2b9d7008cc55dd03021baf53b7c10c4e3e7b6474b8f6215f3 WatchSource:0}: Error finding container 1eac5295d6cb6eb2b9d7008cc55dd03021baf53b7c10c4e3e7b6474b8f6215f3: Status 404 returned error can't find the container with id 1eac5295d6cb6eb2b9d7008cc55dd03021baf53b7c10c4e3e7b6474b8f6215f3
Sep 12 00:33:58.564670 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:58.564635    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-55c4s" event={"ID":"df843273-bc32-44d4-95bd-0ed4b1db206c","Type":"ContainerStarted","Data":"1eac5295d6cb6eb2b9d7008cc55dd03021baf53b7c10c4e3e7b6474b8f6215f3"}
Sep 12 00:33:58.564841 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:58.564647    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:33:58.572354 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:58.572327    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:33:58.650235 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:58.650146    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-bound-sa-token\") pod \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") "
Sep 12 00:33:58.650235 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:58.650220    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rtmq\" (UniqueName: \"kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-kube-api-access-8rtmq\") pod \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") "
Sep 12 00:33:58.650457 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:58.650318    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-image-registry-private-configuration\") pod \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") "
Sep 12 00:33:58.650457 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:58.650361    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-ca-trust-extracted\") pod \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") "
Sep 12 00:33:58.650457 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:58.650389    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-certificates\") pod \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") "
Sep 12 00:33:58.650457 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:58.650423    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-installation-pull-secrets\") pod \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") "
Sep 12 00:33:58.650457 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:58.650451    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-trusted-ca\") pod \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\" (UID: \"dbe5b08b-127a-4b6c-90db-7abd802f6e9a\") "
Sep 12 00:33:58.650704 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:58.650626    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "dbe5b08b-127a-4b6c-90db-7abd802f6e9a" (UID: "dbe5b08b-127a-4b6c-90db-7abd802f6e9a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:33:58.650767 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:58.650736    2558 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-ca-trust-extracted\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:33:58.650840 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:58.650767    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "dbe5b08b-127a-4b6c-90db-7abd802f6e9a" (UID: "dbe5b08b-127a-4b6c-90db-7abd802f6e9a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:33:58.650992 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:58.650935    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "dbe5b08b-127a-4b6c-90db-7abd802f6e9a" (UID: "dbe5b08b-127a-4b6c-90db-7abd802f6e9a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:33:58.652648 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:58.652618    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-kube-api-access-8rtmq" (OuterVolumeSpecName: "kube-api-access-8rtmq") pod "dbe5b08b-127a-4b6c-90db-7abd802f6e9a" (UID: "dbe5b08b-127a-4b6c-90db-7abd802f6e9a"). InnerVolumeSpecName "kube-api-access-8rtmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:33:58.652784 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:58.652730    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "dbe5b08b-127a-4b6c-90db-7abd802f6e9a" (UID: "dbe5b08b-127a-4b6c-90db-7abd802f6e9a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:33:58.653019 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:58.653001    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "dbe5b08b-127a-4b6c-90db-7abd802f6e9a" (UID: "dbe5b08b-127a-4b6c-90db-7abd802f6e9a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:33:58.653079 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:58.653016    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "dbe5b08b-127a-4b6c-90db-7abd802f6e9a" (UID: "dbe5b08b-127a-4b6c-90db-7abd802f6e9a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:33:58.751602 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:58.751566    2558 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-bound-sa-token\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:33:58.751602 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:58.751591    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rtmq\" (UniqueName: \"kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-kube-api-access-8rtmq\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:33:58.751602 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:58.751603    2558 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-image-registry-private-configuration\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:33:58.751602 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:58.751612    2558 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-certificates\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:33:58.751860 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:58.751621    2558 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-installation-pull-secrets\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:33:58.751860 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:58.751629    2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-trusted-ca\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:33:59.208832 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:59.208796    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:59.257422 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:59.257380    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:33:59.568152 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:59.568055    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-599f5d46cb-zk8ln"
Sep 12 00:33:59.599044 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:59.599009    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-599f5d46cb-zk8ln"]
Sep 12 00:33:59.602270 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:59.602244    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-599f5d46cb-zk8ln"]
Sep 12 00:33:59.660659 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:59.660627    2558 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dbe5b08b-127a-4b6c-90db-7abd802f6e9a-registry-tls\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:33:59.736328 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:33:59.736292    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbe5b08b-127a-4b6c-90db-7abd802f6e9a" path="/var/lib/kubelet/pods/dbe5b08b-127a-4b6c-90db-7abd802f6e9a/volumes"
Sep 12 00:34:00.712344 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:00.712306    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:00.739631 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:00.739603    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-nvmpr"]
Sep 12 00:34:00.742800 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:00.742783    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-nvmpr"
Sep 12 00:34:00.745357 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:00.745331    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 12 00:34:00.745490 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:00.745339    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 12 00:34:00.745490 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:00.745385    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-k56m5\""
Sep 12 00:34:00.745490 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:00.745392    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 12 00:34:00.745719 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:00.745704    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 12 00:34:00.745758 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:00.745732    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 12 00:34:00.752381 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:00.752329    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-nvmpr"]
Sep 12 00:34:00.774413 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:00.774381    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:00.872075 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:00.872039    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73009400-1d5f-4778-8a76-50b67d0860b4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-nvmpr\" (UID: \"73009400-1d5f-4778-8a76-50b67d0860b4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nvmpr"
Sep 12 00:34:00.872370 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:00.872102    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lksxz\" (UniqueName: \"kubernetes.io/projected/73009400-1d5f-4778-8a76-50b67d0860b4-kube-api-access-lksxz\") pod \"prometheus-operator-68bcd7f987-nvmpr\" (UID: \"73009400-1d5f-4778-8a76-50b67d0860b4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nvmpr"
Sep 12 00:34:00.872370 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:00.872265    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73009400-1d5f-4778-8a76-50b67d0860b4-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-nvmpr\" (UID: \"73009400-1d5f-4778-8a76-50b67d0860b4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nvmpr"
Sep 12 00:34:00.872370 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:00.872312    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/73009400-1d5f-4778-8a76-50b67d0860b4-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-nvmpr\" (UID: \"73009400-1d5f-4778-8a76-50b67d0860b4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nvmpr"
Sep 12 00:34:00.973435 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:00.973342    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73009400-1d5f-4778-8a76-50b67d0860b4-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-nvmpr\" (UID: \"73009400-1d5f-4778-8a76-50b67d0860b4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nvmpr"
Sep 12 00:34:00.973435 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:00.973385    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/73009400-1d5f-4778-8a76-50b67d0860b4-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-nvmpr\" (UID: \"73009400-1d5f-4778-8a76-50b67d0860b4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nvmpr"
Sep 12 00:34:00.973660 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:00.973438    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73009400-1d5f-4778-8a76-50b67d0860b4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-nvmpr\" (UID: \"73009400-1d5f-4778-8a76-50b67d0860b4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nvmpr"
Sep 12 00:34:00.973660 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:00.973471    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lksxz\" (UniqueName: \"kubernetes.io/projected/73009400-1d5f-4778-8a76-50b67d0860b4-kube-api-access-lksxz\") pod \"prometheus-operator-68bcd7f987-nvmpr\" (UID: \"73009400-1d5f-4778-8a76-50b67d0860b4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nvmpr"
Sep 12 00:34:00.973660 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:34:00.973593    2558 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 12 00:34:00.973838 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:34:00.973677    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73009400-1d5f-4778-8a76-50b67d0860b4-prometheus-operator-tls podName:73009400-1d5f-4778-8a76-50b67d0860b4 nodeName:}" failed. No retries permitted until 2026-09-12 00:34:01.473654736 +0000 UTC m=+278.147713588 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/73009400-1d5f-4778-8a76-50b67d0860b4-prometheus-operator-tls") pod "prometheus-operator-68bcd7f987-nvmpr" (UID: "73009400-1d5f-4778-8a76-50b67d0860b4") : secret "prometheus-operator-tls" not found
Sep 12 00:34:00.974130 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:00.974107    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73009400-1d5f-4778-8a76-50b67d0860b4-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-nvmpr\" (UID: \"73009400-1d5f-4778-8a76-50b67d0860b4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nvmpr"
Sep 12 00:34:00.976299 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:00.976279    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73009400-1d5f-4778-8a76-50b67d0860b4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-nvmpr\" (UID: \"73009400-1d5f-4778-8a76-50b67d0860b4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nvmpr"
Sep 12 00:34:00.996158 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:00.996112    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lksxz\" (UniqueName: \"kubernetes.io/projected/73009400-1d5f-4778-8a76-50b67d0860b4-kube-api-access-lksxz\") pod \"prometheus-operator-68bcd7f987-nvmpr\" (UID: \"73009400-1d5f-4778-8a76-50b67d0860b4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nvmpr"
Sep 12 00:34:01.364669 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:01.364575    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:01.478104 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:01.478048    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/73009400-1d5f-4778-8a76-50b67d0860b4-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-nvmpr\" (UID: \"73009400-1d5f-4778-8a76-50b67d0860b4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nvmpr"
Sep 12 00:34:01.480539 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:01.480515    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/73009400-1d5f-4778-8a76-50b67d0860b4-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-nvmpr\" (UID: \"73009400-1d5f-4778-8a76-50b67d0860b4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nvmpr"
Sep 12 00:34:01.652695 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:01.652619    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-nvmpr"
Sep 12 00:34:01.933515 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:01.932591    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-nvmpr"]
Sep 12 00:34:02.578712 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:02.578679    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-nvmpr" event={"ID":"73009400-1d5f-4778-8a76-50b67d0860b4","Type":"ContainerStarted","Data":"3e2bd2d372c62c381c08514424d5e85d832b6bd75a7c59bbf3a5c6fb15c52c3c"}
Sep 12 00:34:03.583264 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:03.583228    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-nvmpr" event={"ID":"73009400-1d5f-4778-8a76-50b67d0860b4","Type":"ContainerStarted","Data":"3fb2b04784b5ef236504818599a42f95543a8d60819b9b7450de955158935f42"}
Sep 12 00:34:03.583618 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:03.583296    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-nvmpr" event={"ID":"73009400-1d5f-4778-8a76-50b67d0860b4","Type":"ContainerStarted","Data":"d012d66d0c5f3cb1a636b59541da5e7cc0dd87636ca31c0ca40b002a470f7d03"}
Sep 12 00:34:03.599268 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:03.599218    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-nvmpr" podStartSLOduration=2.108461718 podStartE2EDuration="3.599200574s" totalImagesPullingTime="1.490738856s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:34:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:34:03.598661283 +0000 UTC m=+280.272720143" watchObservedRunningTime="2026-09-12 00:34:03.599200574 +0000 UTC m=+280.273259433"
Sep 12 00:34:03.709479 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:03.709433    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:04.836555 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:04.836493    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:05.060170 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.060126    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:05.092791 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.092700    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6mxtl"]
Sep 12 00:34:05.095917 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.095894    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6mxtl"
Sep 12 00:34:05.098354 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.098329    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 12 00:34:05.098488 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.098471    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 12 00:34:05.098753 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.098738    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rm5ll\""
Sep 12 00:34:05.098842 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.098772    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 12 00:34:05.139418 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.139368    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:05.198535 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.198489    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:05.207641 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.207611    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1d102e18-0228-4bb0-bc60-6c2942d7b321-root\") pod \"node-exporter-6mxtl\" (UID: \"1d102e18-0228-4bb0-bc60-6c2942d7b321\") " pod="openshift-monitoring/node-exporter-6mxtl"
Sep 12 00:34:05.207641 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.207644    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1d102e18-0228-4bb0-bc60-6c2942d7b321-metrics-client-ca\") pod \"node-exporter-6mxtl\" (UID: \"1d102e18-0228-4bb0-bc60-6c2942d7b321\") " pod="openshift-monitoring/node-exporter-6mxtl"
Sep 12 00:34:05.207816 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.207713    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1d102e18-0228-4bb0-bc60-6c2942d7b321-sys\") pod \"node-exporter-6mxtl\" (UID: \"1d102e18-0228-4bb0-bc60-6c2942d7b321\") " pod="openshift-monitoring/node-exporter-6mxtl"
Sep 12 00:34:05.207816 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.207748    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1d102e18-0228-4bb0-bc60-6c2942d7b321-node-exporter-tls\") pod \"node-exporter-6mxtl\" (UID: \"1d102e18-0228-4bb0-bc60-6c2942d7b321\") " pod="openshift-monitoring/node-exporter-6mxtl"
Sep 12 00:34:05.207816 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.207792    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1d102e18-0228-4bb0-bc60-6c2942d7b321-node-exporter-textfile\") pod \"node-exporter-6mxtl\" (UID: \"1d102e18-0228-4bb0-bc60-6c2942d7b321\") " pod="openshift-monitoring/node-exporter-6mxtl"
Sep 12 00:34:05.207816 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.207812    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1d102e18-0228-4bb0-bc60-6c2942d7b321-node-exporter-accelerators-collector-config\") pod \"node-exporter-6mxtl\" (UID: \"1d102e18-0228-4bb0-bc60-6c2942d7b321\") " pod="openshift-monitoring/node-exporter-6mxtl"
Sep 12 00:34:05.207939 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.207837    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1d102e18-0228-4bb0-bc60-6c2942d7b321-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6mxtl\" (UID: \"1d102e18-0228-4bb0-bc60-6c2942d7b321\") " pod="openshift-monitoring/node-exporter-6mxtl"
Sep 12 00:34:05.207939 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.207867    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1d102e18-0228-4bb0-bc60-6c2942d7b321-node-exporter-wtmp\") pod \"node-exporter-6mxtl\" (UID: \"1d102e18-0228-4bb0-bc60-6c2942d7b321\") " pod="openshift-monitoring/node-exporter-6mxtl"
Sep 12 00:34:05.207939 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.207881    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68fg5\" (UniqueName: \"kubernetes.io/projected/1d102e18-0228-4bb0-bc60-6c2942d7b321-kube-api-access-68fg5\") pod \"node-exporter-6mxtl\" (UID: \"1d102e18-0228-4bb0-bc60-6c2942d7b321\") " pod="openshift-monitoring/node-exporter-6mxtl"
Sep 12 00:34:05.224738 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:34:05.224699    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-5wzpq" podUID="f0b419ed-e351-402a-9b9b-ff1534048f68"
Sep 12 00:34:05.297691 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.297651    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:05.308748 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.308716    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1d102e18-0228-4bb0-bc60-6c2942d7b321-node-exporter-wtmp\") pod \"node-exporter-6mxtl\" (UID: \"1d102e18-0228-4bb0-bc60-6c2942d7b321\") " pod="openshift-monitoring/node-exporter-6mxtl"
Sep 12 00:34:05.308748 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.308747    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68fg5\" (UniqueName: \"kubernetes.io/projected/1d102e18-0228-4bb0-bc60-6c2942d7b321-kube-api-access-68fg5\") pod \"node-exporter-6mxtl\" (UID: \"1d102e18-0228-4bb0-bc60-6c2942d7b321\") " pod="openshift-monitoring/node-exporter-6mxtl"
Sep 12 00:34:05.308961 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.308776    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1d102e18-0228-4bb0-bc60-6c2942d7b321-root\") pod \"node-exporter-6mxtl\" (UID: \"1d102e18-0228-4bb0-bc60-6c2942d7b321\") " pod="openshift-monitoring/node-exporter-6mxtl"
Sep 12 00:34:05.308961 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.308807    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1d102e18-0228-4bb0-bc60-6c2942d7b321-metrics-client-ca\") pod \"node-exporter-6mxtl\" (UID: \"1d102e18-0228-4bb0-bc60-6c2942d7b321\") " pod="openshift-monitoring/node-exporter-6mxtl"
Sep 12 00:34:05.308961 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.308927    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1d102e18-0228-4bb0-bc60-6c2942d7b321-root\") pod \"node-exporter-6mxtl\" (UID: \"1d102e18-0228-4bb0-bc60-6c2942d7b321\") " pod="openshift-monitoring/node-exporter-6mxtl"
Sep 12 00:34:05.308961 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.308927    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1d102e18-0228-4bb0-bc60-6c2942d7b321-node-exporter-wtmp\") pod \"node-exporter-6mxtl\" (UID: \"1d102e18-0228-4bb0-bc60-6c2942d7b321\") " pod="openshift-monitoring/node-exporter-6mxtl"
Sep 12 00:34:05.308961 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.308955    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1d102e18-0228-4bb0-bc60-6c2942d7b321-sys\") pod \"node-exporter-6mxtl\" (UID: \"1d102e18-0228-4bb0-bc60-6c2942d7b321\") " pod="openshift-monitoring/node-exporter-6mxtl"
Sep 12 00:34:05.309238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.308987    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1d102e18-0228-4bb0-bc60-6c2942d7b321-sys\") pod \"node-exporter-6mxtl\" (UID: \"1d102e18-0228-4bb0-bc60-6c2942d7b321\") " pod="openshift-monitoring/node-exporter-6mxtl"
Sep 12 00:34:05.309238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.308992    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1d102e18-0228-4bb0-bc60-6c2942d7b321-node-exporter-tls\") pod \"node-exporter-6mxtl\" (UID: \"1d102e18-0228-4bb0-bc60-6c2942d7b321\") " pod="openshift-monitoring/node-exporter-6mxtl"
Sep 12 00:34:05.309238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.309042    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1d102e18-0228-4bb0-bc60-6c2942d7b321-node-exporter-textfile\") pod \"node-exporter-6mxtl\" (UID: \"1d102e18-0228-4bb0-bc60-6c2942d7b321\") " pod="openshift-monitoring/node-exporter-6mxtl"
Sep 12 00:34:05.309238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.309072    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1d102e18-0228-4bb0-bc60-6c2942d7b321-node-exporter-accelerators-collector-config\") pod \"node-exporter-6mxtl\" (UID: \"1d102e18-0228-4bb0-bc60-6c2942d7b321\") " pod="openshift-monitoring/node-exporter-6mxtl"
Sep 12 00:34:05.309238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.309122    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1d102e18-0228-4bb0-bc60-6c2942d7b321-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6mxtl\" (UID: \"1d102e18-0228-4bb0-bc60-6c2942d7b321\") " pod="openshift-monitoring/node-exporter-6mxtl"
Sep 12 00:34:05.309475 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.309427    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1d102e18-0228-4bb0-bc60-6c2942d7b321-node-exporter-textfile\") pod \"node-exporter-6mxtl\" (UID: \"1d102e18-0228-4bb0-bc60-6c2942d7b321\") " pod="openshift-monitoring/node-exporter-6mxtl"
Sep 12 00:34:05.309524 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.309491    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1d102e18-0228-4bb0-bc60-6c2942d7b321-metrics-client-ca\") pod \"node-exporter-6mxtl\" (UID: \"1d102e18-0228-4bb0-bc60-6c2942d7b321\") " pod="openshift-monitoring/node-exporter-6mxtl"
Sep 12 00:34:05.309638 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.309619    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1d102e18-0228-4bb0-bc60-6c2942d7b321-node-exporter-accelerators-collector-config\") pod \"node-exporter-6mxtl\" (UID: \"1d102e18-0228-4bb0-bc60-6c2942d7b321\") " pod="openshift-monitoring/node-exporter-6mxtl"
Sep 12 00:34:05.311826 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.311794    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1d102e18-0228-4bb0-bc60-6c2942d7b321-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6mxtl\" (UID: \"1d102e18-0228-4bb0-bc60-6c2942d7b321\") " pod="openshift-monitoring/node-exporter-6mxtl"
Sep 12 00:34:05.311956 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.311933    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1d102e18-0228-4bb0-bc60-6c2942d7b321-node-exporter-tls\") pod \"node-exporter-6mxtl\" (UID: \"1d102e18-0228-4bb0-bc60-6c2942d7b321\") " pod="openshift-monitoring/node-exporter-6mxtl"
Sep 12 00:34:05.316310 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.316283    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68fg5\" (UniqueName: \"kubernetes.io/projected/1d102e18-0228-4bb0-bc60-6c2942d7b321-kube-api-access-68fg5\") pod \"node-exporter-6mxtl\" (UID: \"1d102e18-0228-4bb0-bc60-6c2942d7b321\") " pod="openshift-monitoring/node-exporter-6mxtl"
Sep 12 00:34:05.399975 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.399897    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:05.407572 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.407527    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6mxtl"
Sep 12 00:34:05.430257 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:34:05.430225    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d102e18_0228_4bb0_bc60_6c2942d7b321.slice/crio-8b024ac7c92156b4e526cd4a3b65829505450b5b2844add5d9c9490450042fe1 WatchSource:0}: Error finding container 8b024ac7c92156b4e526cd4a3b65829505450b5b2844add5d9c9490450042fe1: Status 404 returned error can't find the container with id 8b024ac7c92156b4e526cd4a3b65829505450b5b2844add5d9c9490450042fe1
Sep 12 00:34:05.500052 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.499979    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:05.591750 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.591709    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6mxtl" event={"ID":"1d102e18-0228-4bb0-bc60-6c2942d7b321","Type":"ContainerStarted","Data":"8b024ac7c92156b4e526cd4a3b65829505450b5b2844add5d9c9490450042fe1"}
Sep 12 00:34:05.591942 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.591724    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5wzpq"
Sep 12 00:34:05.686744 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:05.686715    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:06.063817 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:06.063743    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:06.136043 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:06.136006    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:06.947301 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:06.947267    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66665b5885-wbfxh"]
Sep 12 00:34:06.955963 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:06.955845    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66665b5885-wbfxh"
Sep 12 00:34:06.959508 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:06.959478    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66665b5885-wbfxh"]
Sep 12 00:34:06.959842 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:06.959802    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 12 00:34:06.960423 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:06.960401    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 12 00:34:06.960634 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:06.960606    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 12 00:34:06.960883 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:06.960860    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 12 00:34:06.961352 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:06.961181    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 12 00:34:06.961567 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:06.961462    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-wfdtk\""
Sep 12 00:34:07.027023 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.026939    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3bba313c-7ee4-46de-b7bf-47e15ebb1add-service-ca\") pod \"console-66665b5885-wbfxh\" (UID: \"3bba313c-7ee4-46de-b7bf-47e15ebb1add\") " pod="openshift-console/console-66665b5885-wbfxh"
Sep 12 00:34:07.027023 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.026976    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btnr4\" (UniqueName: \"kubernetes.io/projected/3bba313c-7ee4-46de-b7bf-47e15ebb1add-kube-api-access-btnr4\") pod \"console-66665b5885-wbfxh\" (UID: \"3bba313c-7ee4-46de-b7bf-47e15ebb1add\") " pod="openshift-console/console-66665b5885-wbfxh"
Sep 12 00:34:07.027023 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.027009    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3bba313c-7ee4-46de-b7bf-47e15ebb1add-console-oauth-config\") pod \"console-66665b5885-wbfxh\" (UID: \"3bba313c-7ee4-46de-b7bf-47e15ebb1add\") " pod="openshift-console/console-66665b5885-wbfxh"
Sep 12 00:34:07.027277 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.027117    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3bba313c-7ee4-46de-b7bf-47e15ebb1add-console-config\") pod \"console-66665b5885-wbfxh\" (UID: \"3bba313c-7ee4-46de-b7bf-47e15ebb1add\") " pod="openshift-console/console-66665b5885-wbfxh"
Sep 12 00:34:07.027277 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.027137    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3bba313c-7ee4-46de-b7bf-47e15ebb1add-oauth-serving-cert\") pod \"console-66665b5885-wbfxh\" (UID: \"3bba313c-7ee4-46de-b7bf-47e15ebb1add\") " pod="openshift-console/console-66665b5885-wbfxh"
Sep 12 00:34:07.027277 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.027152    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3bba313c-7ee4-46de-b7bf-47e15ebb1add-console-serving-cert\") pod \"console-66665b5885-wbfxh\" (UID: \"3bba313c-7ee4-46de-b7bf-47e15ebb1add\") " pod="openshift-console/console-66665b5885-wbfxh"
Sep 12 00:34:07.056960 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.056920    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd"]
Sep 12 00:34:07.078412 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.078372    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd"]
Sep 12 00:34:07.078821 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.078484    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd"
Sep 12 00:34:07.081619 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.081125    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 12 00:34:07.081619 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.081332    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-vrd7v\""
Sep 12 00:34:07.081619 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.081387    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 12 00:34:07.081619 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.081490    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-gj9dbv7kj2tv\""
Sep 12 00:34:07.081904 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.081712    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 12 00:34:07.081904 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.081902    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 12 00:34:07.082119 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.082078    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 12 00:34:07.128250 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.128216    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3bba313c-7ee4-46de-b7bf-47e15ebb1add-console-config\") pod \"console-66665b5885-wbfxh\" (UID: \"3bba313c-7ee4-46de-b7bf-47e15ebb1add\") " pod="openshift-console/console-66665b5885-wbfxh"
Sep 12 00:34:07.128445 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.128269    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3bba313c-7ee4-46de-b7bf-47e15ebb1add-oauth-serving-cert\") pod \"console-66665b5885-wbfxh\" (UID: \"3bba313c-7ee4-46de-b7bf-47e15ebb1add\") " pod="openshift-console/console-66665b5885-wbfxh"
Sep 12 00:34:07.128445 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.128320    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3bba313c-7ee4-46de-b7bf-47e15ebb1add-console-serving-cert\") pod \"console-66665b5885-wbfxh\" (UID: \"3bba313c-7ee4-46de-b7bf-47e15ebb1add\") " pod="openshift-console/console-66665b5885-wbfxh"
Sep 12 00:34:07.128445 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.128373    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3bba313c-7ee4-46de-b7bf-47e15ebb1add-service-ca\") pod \"console-66665b5885-wbfxh\" (UID: \"3bba313c-7ee4-46de-b7bf-47e15ebb1add\") " pod="openshift-console/console-66665b5885-wbfxh"
Sep 12 00:34:07.128445 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.128410    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btnr4\" (UniqueName: \"kubernetes.io/projected/3bba313c-7ee4-46de-b7bf-47e15ebb1add-kube-api-access-btnr4\") pod \"console-66665b5885-wbfxh\" (UID: \"3bba313c-7ee4-46de-b7bf-47e15ebb1add\") " pod="openshift-console/console-66665b5885-wbfxh"
Sep 12 00:34:07.128654 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.128454    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3bba313c-7ee4-46de-b7bf-47e15ebb1add-console-oauth-config\") pod \"console-66665b5885-wbfxh\" (UID: \"3bba313c-7ee4-46de-b7bf-47e15ebb1add\") " pod="openshift-console/console-66665b5885-wbfxh"
Sep 12 00:34:07.129120 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.129055    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3bba313c-7ee4-46de-b7bf-47e15ebb1add-oauth-serving-cert\") pod \"console-66665b5885-wbfxh\" (UID: \"3bba313c-7ee4-46de-b7bf-47e15ebb1add\") " pod="openshift-console/console-66665b5885-wbfxh"
Sep 12 00:34:07.129120 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.129055    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3bba313c-7ee4-46de-b7bf-47e15ebb1add-console-config\") pod \"console-66665b5885-wbfxh\" (UID: \"3bba313c-7ee4-46de-b7bf-47e15ebb1add\") " pod="openshift-console/console-66665b5885-wbfxh"
Sep 12 00:34:07.129303 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.129276    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3bba313c-7ee4-46de-b7bf-47e15ebb1add-service-ca\") pod \"console-66665b5885-wbfxh\" (UID: \"3bba313c-7ee4-46de-b7bf-47e15ebb1add\") " pod="openshift-console/console-66665b5885-wbfxh"
Sep 12 00:34:07.131644 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.131622    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3bba313c-7ee4-46de-b7bf-47e15ebb1add-console-oauth-config\") pod \"console-66665b5885-wbfxh\" (UID: \"3bba313c-7ee4-46de-b7bf-47e15ebb1add\") " pod="openshift-console/console-66665b5885-wbfxh"
Sep 12 00:34:07.131644 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.131635    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3bba313c-7ee4-46de-b7bf-47e15ebb1add-console-serving-cert\") pod \"console-66665b5885-wbfxh\" (UID: \"3bba313c-7ee4-46de-b7bf-47e15ebb1add\") " pod="openshift-console/console-66665b5885-wbfxh"
Sep 12 00:34:07.137023 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.137000    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btnr4\" (UniqueName: \"kubernetes.io/projected/3bba313c-7ee4-46de-b7bf-47e15ebb1add-kube-api-access-btnr4\") pod \"console-66665b5885-wbfxh\" (UID: \"3bba313c-7ee4-46de-b7bf-47e15ebb1add\") " pod="openshift-console/console-66665b5885-wbfxh"
Sep 12 00:34:07.146627 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.146597    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:07.229512 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.229474    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5f89c9a5-1e8c-4e54-be92-c104eb70007f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-84d9f6d8bf-grjpd\" (UID: \"5f89c9a5-1e8c-4e54-be92-c104eb70007f\") " pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd"
Sep 12 00:34:07.229712 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.229521    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5f89c9a5-1e8c-4e54-be92-c104eb70007f-secret-thanos-querier-tls\") pod \"thanos-querier-84d9f6d8bf-grjpd\" (UID: \"5f89c9a5-1e8c-4e54-be92-c104eb70007f\") " pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd"
Sep 12 00:34:07.229712 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.229547    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5f89c9a5-1e8c-4e54-be92-c104eb70007f-secret-grpc-tls\") pod \"thanos-querier-84d9f6d8bf-grjpd\" (UID: \"5f89c9a5-1e8c-4e54-be92-c104eb70007f\") " pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd"
Sep 12 00:34:07.229712 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.229638    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f89c9a5-1e8c-4e54-be92-c104eb70007f-metrics-client-ca\") pod \"thanos-querier-84d9f6d8bf-grjpd\" (UID: \"5f89c9a5-1e8c-4e54-be92-c104eb70007f\") " pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd"
Sep 12 00:34:07.229712 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.229682    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5f89c9a5-1e8c-4e54-be92-c104eb70007f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-84d9f6d8bf-grjpd\" (UID: \"5f89c9a5-1e8c-4e54-be92-c104eb70007f\") " pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd"
Sep 12 00:34:07.229712 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.229706    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsgsj\" (UniqueName: \"kubernetes.io/projected/5f89c9a5-1e8c-4e54-be92-c104eb70007f-kube-api-access-qsgsj\") pod \"thanos-querier-84d9f6d8bf-grjpd\" (UID: \"5f89c9a5-1e8c-4e54-be92-c104eb70007f\") " pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd"
Sep 12 00:34:07.229906 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.229735    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5f89c9a5-1e8c-4e54-be92-c104eb70007f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-84d9f6d8bf-grjpd\" (UID: \"5f89c9a5-1e8c-4e54-be92-c104eb70007f\") " pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd"
Sep 12 00:34:07.229906 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.229876    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5f89c9a5-1e8c-4e54-be92-c104eb70007f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-84d9f6d8bf-grjpd\" (UID: \"5f89c9a5-1e8c-4e54-be92-c104eb70007f\") " pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd"
Sep 12 00:34:07.310038 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.309949    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66665b5885-wbfxh"
Sep 12 00:34:07.331054 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.331017    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5f89c9a5-1e8c-4e54-be92-c104eb70007f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-84d9f6d8bf-grjpd\" (UID: \"5f89c9a5-1e8c-4e54-be92-c104eb70007f\") " pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd"
Sep 12 00:34:07.331202 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.331068    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsgsj\" (UniqueName: \"kubernetes.io/projected/5f89c9a5-1e8c-4e54-be92-c104eb70007f-kube-api-access-qsgsj\") pod \"thanos-querier-84d9f6d8bf-grjpd\" (UID: \"5f89c9a5-1e8c-4e54-be92-c104eb70007f\") " pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd"
Sep 12 00:34:07.331202 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.331169    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5f89c9a5-1e8c-4e54-be92-c104eb70007f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-84d9f6d8bf-grjpd\" (UID: \"5f89c9a5-1e8c-4e54-be92-c104eb70007f\") " pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd"
Sep 12 00:34:07.331334 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.331288    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5f89c9a5-1e8c-4e54-be92-c104eb70007f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-84d9f6d8bf-grjpd\" (UID: \"5f89c9a5-1e8c-4e54-be92-c104eb70007f\") " pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd"
Sep 12 00:34:07.331387 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.331330    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5f89c9a5-1e8c-4e54-be92-c104eb70007f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-84d9f6d8bf-grjpd\" (UID: \"5f89c9a5-1e8c-4e54-be92-c104eb70007f\") " pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd"
Sep 12 00:34:07.331387 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.331359    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5f89c9a5-1e8c-4e54-be92-c104eb70007f-secret-thanos-querier-tls\") pod \"thanos-querier-84d9f6d8bf-grjpd\" (UID: \"5f89c9a5-1e8c-4e54-be92-c104eb70007f\") " pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd"
Sep 12 00:34:07.331490 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.331386    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5f89c9a5-1e8c-4e54-be92-c104eb70007f-secret-grpc-tls\") pod \"thanos-querier-84d9f6d8bf-grjpd\" (UID: \"5f89c9a5-1e8c-4e54-be92-c104eb70007f\") " pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd"
Sep 12 00:34:07.331490 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.331447    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f89c9a5-1e8c-4e54-be92-c104eb70007f-metrics-client-ca\") pod \"thanos-querier-84d9f6d8bf-grjpd\" (UID: \"5f89c9a5-1e8c-4e54-be92-c104eb70007f\") " pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd"
Sep 12 00:34:07.332395 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.332348    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f89c9a5-1e8c-4e54-be92-c104eb70007f-metrics-client-ca\") pod \"thanos-querier-84d9f6d8bf-grjpd\" (UID: \"5f89c9a5-1e8c-4e54-be92-c104eb70007f\") " pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd"
Sep 12 00:34:07.334514 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.334464    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5f89c9a5-1e8c-4e54-be92-c104eb70007f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-84d9f6d8bf-grjpd\" (UID: \"5f89c9a5-1e8c-4e54-be92-c104eb70007f\") " pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd"
Sep 12 00:34:07.334977 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.334951    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5f89c9a5-1e8c-4e54-be92-c104eb70007f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-84d9f6d8bf-grjpd\" (UID: \"5f89c9a5-1e8c-4e54-be92-c104eb70007f\") " pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd"
Sep 12 00:34:07.335230 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.335182    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5f89c9a5-1e8c-4e54-be92-c104eb70007f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-84d9f6d8bf-grjpd\" (UID: \"5f89c9a5-1e8c-4e54-be92-c104eb70007f\") " pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd"
Sep 12 00:34:07.335350 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.335231    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5f89c9a5-1e8c-4e54-be92-c104eb70007f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-84d9f6d8bf-grjpd\" (UID: \"5f89c9a5-1e8c-4e54-be92-c104eb70007f\") " pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd"
Sep 12 00:34:07.336316 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.336292    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5f89c9a5-1e8c-4e54-be92-c104eb70007f-secret-grpc-tls\") pod \"thanos-querier-84d9f6d8bf-grjpd\" (UID: \"5f89c9a5-1e8c-4e54-be92-c104eb70007f\") " pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd"
Sep 12 00:34:07.336635 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.336614    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5f89c9a5-1e8c-4e54-be92-c104eb70007f-secret-thanos-querier-tls\") pod \"thanos-querier-84d9f6d8bf-grjpd\" (UID: \"5f89c9a5-1e8c-4e54-be92-c104eb70007f\") " pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd"
Sep 12 00:34:07.338448 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.338427    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsgsj\" (UniqueName: \"kubernetes.io/projected/5f89c9a5-1e8c-4e54-be92-c104eb70007f-kube-api-access-qsgsj\") pod \"thanos-querier-84d9f6d8bf-grjpd\" (UID: \"5f89c9a5-1e8c-4e54-be92-c104eb70007f\") " pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd"
Sep 12 00:34:07.389889 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.389856    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd"
Sep 12 00:34:07.588565 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.588493    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66665b5885-wbfxh"]
Sep 12 00:34:07.591228 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:34:07.591198    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bba313c_7ee4_46de_b7bf_47e15ebb1add.slice/crio-91e18c6a1703eae6c199d16baba1425f8a436b400600078b836d2ef1f686a937 WatchSource:0}: Error finding container 91e18c6a1703eae6c199d16baba1425f8a436b400600078b836d2ef1f686a937: Status 404 returned error can't find the container with id 91e18c6a1703eae6c199d16baba1425f8a436b400600078b836d2ef1f686a937
Sep 12 00:34:07.601213 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.601179    2558 generic.go:356] "Generic (PLEG): container finished" podID="1d102e18-0228-4bb0-bc60-6c2942d7b321" containerID="a49204a11de9fa7ee1bd039803a2a301be841019925d911844b8e805afadc964" exitCode=0
Sep 12 00:34:07.601336 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.601278    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6mxtl" event={"ID":"1d102e18-0228-4bb0-bc60-6c2942d7b321","Type":"ContainerDied","Data":"a49204a11de9fa7ee1bd039803a2a301be841019925d911844b8e805afadc964"}
Sep 12 00:34:07.602700 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.602648    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66665b5885-wbfxh" event={"ID":"3bba313c-7ee4-46de-b7bf-47e15ebb1add","Type":"ContainerStarted","Data":"91e18c6a1703eae6c199d16baba1425f8a436b400600078b836d2ef1f686a937"}
Sep 12 00:34:07.665115 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.665074    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd"]
Sep 12 00:34:07.668735 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:34:07.668705    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f89c9a5_1e8c_4e54_be92_c104eb70007f.slice/crio-1014524f4d0fa0281b0bc456638a4347df48fe868e05725f687de6a72effbc74 WatchSource:0}: Error finding container 1014524f4d0fa0281b0bc456638a4347df48fe868e05725f687de6a72effbc74: Status 404 returned error can't find the container with id 1014524f4d0fa0281b0bc456638a4347df48fe868e05725f687de6a72effbc74
Sep 12 00:34:07.997354 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:07.997323    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:08.045010 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:08.044975    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:08.137963 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:08.137932    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0b419ed-e351-402a-9b9b-ff1534048f68-metrics-tls\") pod \"dns-default-5wzpq\" (UID: \"f0b419ed-e351-402a-9b9b-ff1534048f68\") " pod="openshift-dns/dns-default-5wzpq"
Sep 12 00:34:08.140606 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:08.140580    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0b419ed-e351-402a-9b9b-ff1534048f68-metrics-tls\") pod \"dns-default-5wzpq\" (UID: \"f0b419ed-e351-402a-9b9b-ff1534048f68\") " pod="openshift-dns/dns-default-5wzpq"
Sep 12 00:34:08.187740 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:08.187657    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:08.281494 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:08.281404    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:08.295298 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:08.295275    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-h7jvg\""
Sep 12 00:34:08.303330 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:08.303302    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5wzpq"
Sep 12 00:34:08.583338 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:08.583238    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5wzpq"]
Sep 12 00:34:08.586643 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:34:08.586438    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0b419ed_e351_402a_9b9b_ff1534048f68.slice/crio-b4d49fb322fd9e697d7a5f35049518344439230201ef20946a778f3b638da4fa WatchSource:0}: Error finding container b4d49fb322fd9e697d7a5f35049518344439230201ef20946a778f3b638da4fa: Status 404 returned error can't find the container with id b4d49fb322fd9e697d7a5f35049518344439230201ef20946a778f3b638da4fa
Sep 12 00:34:08.609446 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:08.609402    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5wzpq" event={"ID":"f0b419ed-e351-402a-9b9b-ff1534048f68","Type":"ContainerStarted","Data":"b4d49fb322fd9e697d7a5f35049518344439230201ef20946a778f3b638da4fa"}
Sep 12 00:34:08.611710 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:08.611671    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd" event={"ID":"5f89c9a5-1e8c-4e54-be92-c104eb70007f","Type":"ContainerStarted","Data":"1014524f4d0fa0281b0bc456638a4347df48fe868e05725f687de6a72effbc74"}
Sep 12 00:34:08.614594 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:08.614566    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6mxtl" event={"ID":"1d102e18-0228-4bb0-bc60-6c2942d7b321","Type":"ContainerStarted","Data":"1b2f4d39b6b0c9338e52d049ac79572b5fdb1f1ed54b8974f9b5511e8c53ae07"}
Sep 12 00:34:08.614711 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:08.614621    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6mxtl" event={"ID":"1d102e18-0228-4bb0-bc60-6c2942d7b321","Type":"ContainerStarted","Data":"f3afa23fa8fadc42026f538d52ddcd078b3e0ba8e67d87b6b99b52ac564107d7"}
Sep 12 00:34:08.634509 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:08.632927    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6mxtl" podStartSLOduration=2.215398901 podStartE2EDuration="3.632904647s" totalImagesPullingTime="1.335606043s" totalInitContainerRuntime="81.899703ms" isStatefulPod=true podCreationTimestamp="2026-09-12 00:34:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:34:08.632502623 +0000 UTC m=+285.306561483" watchObservedRunningTime="2026-09-12 00:34:08.632904647 +0000 UTC m=+285.306963507"
Sep 12 00:34:09.489217 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:09.489180    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-86b96456cd-zvh4c"]
Sep 12 00:34:09.491729 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:09.491704    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-86b96456cd-zvh4c"
Sep 12 00:34:09.494378 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:09.494356    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 12 00:34:09.494577 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:09.494356    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 12 00:34:09.495266 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:09.495229    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 12 00:34:09.495370 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:09.495234    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-qkgj6\""
Sep 12 00:34:09.495598 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:09.495579    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-ffhuk29imf8i8\""
Sep 12 00:34:09.495673 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:09.495579    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 12 00:34:09.501874 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:09.501808    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-86b96456cd-zvh4c"]
Sep 12 00:34:09.649288 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:09.649239    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/da806a2e-7fdf-4973-b36b-b646682281a4-secret-metrics-server-tls\") pod \"metrics-server-86b96456cd-zvh4c\" (UID: \"da806a2e-7fdf-4973-b36b-b646682281a4\") " pod="openshift-monitoring/metrics-server-86b96456cd-zvh4c"
Sep 12 00:34:09.649288 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:09.649291    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/da806a2e-7fdf-4973-b36b-b646682281a4-audit-log\") pod \"metrics-server-86b96456cd-zvh4c\" (UID: \"da806a2e-7fdf-4973-b36b-b646682281a4\") " pod="openshift-monitoring/metrics-server-86b96456cd-zvh4c"
Sep 12 00:34:09.649533 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:09.649311    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/da806a2e-7fdf-4973-b36b-b646682281a4-secret-metrics-server-client-certs\") pod \"metrics-server-86b96456cd-zvh4c\" (UID: \"da806a2e-7fdf-4973-b36b-b646682281a4\") " pod="openshift-monitoring/metrics-server-86b96456cd-zvh4c"
Sep 12 00:34:09.649533 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:09.649352    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/da806a2e-7fdf-4973-b36b-b646682281a4-metrics-server-audit-profiles\") pod \"metrics-server-86b96456cd-zvh4c\" (UID: \"da806a2e-7fdf-4973-b36b-b646682281a4\") " pod="openshift-monitoring/metrics-server-86b96456cd-zvh4c"
Sep 12 00:34:09.649533 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:09.649383    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kpx6\" (UniqueName: \"kubernetes.io/projected/da806a2e-7fdf-4973-b36b-b646682281a4-kube-api-access-5kpx6\") pod \"metrics-server-86b96456cd-zvh4c\" (UID: \"da806a2e-7fdf-4973-b36b-b646682281a4\") " pod="openshift-monitoring/metrics-server-86b96456cd-zvh4c"
Sep 12 00:34:09.649533 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:09.649424    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da806a2e-7fdf-4973-b36b-b646682281a4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-86b96456cd-zvh4c\" (UID: \"da806a2e-7fdf-4973-b36b-b646682281a4\") " pod="openshift-monitoring/metrics-server-86b96456cd-zvh4c"
Sep 12 00:34:09.649533 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:09.649447    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/da806a2e-7fdf-4973-b36b-b646682281a4-client-ca-bundle\") pod \"metrics-server-86b96456cd-zvh4c\" (UID: \"da806a2e-7fdf-4973-b36b-b646682281a4\") " pod="openshift-monitoring/metrics-server-86b96456cd-zvh4c"
Sep 12 00:34:09.750540 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:09.750454    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/da806a2e-7fdf-4973-b36b-b646682281a4-secret-metrics-server-tls\") pod \"metrics-server-86b96456cd-zvh4c\" (UID: \"da806a2e-7fdf-4973-b36b-b646682281a4\") " pod="openshift-monitoring/metrics-server-86b96456cd-zvh4c"
Sep 12 00:34:09.750540 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:09.750517    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/da806a2e-7fdf-4973-b36b-b646682281a4-audit-log\") pod \"metrics-server-86b96456cd-zvh4c\" (UID: \"da806a2e-7fdf-4973-b36b-b646682281a4\") " pod="openshift-monitoring/metrics-server-86b96456cd-zvh4c"
Sep 12 00:34:09.750758 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:09.750546    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/da806a2e-7fdf-4973-b36b-b646682281a4-secret-metrics-server-client-certs\") pod \"metrics-server-86b96456cd-zvh4c\" (UID: \"da806a2e-7fdf-4973-b36b-b646682281a4\") " pod="openshift-monitoring/metrics-server-86b96456cd-zvh4c"
Sep 12 00:34:09.750758 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:09.750620    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/da806a2e-7fdf-4973-b36b-b646682281a4-metrics-server-audit-profiles\") pod \"metrics-server-86b96456cd-zvh4c\" (UID: \"da806a2e-7fdf-4973-b36b-b646682281a4\") " pod="openshift-monitoring/metrics-server-86b96456cd-zvh4c"
Sep 12 00:34:09.750758 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:09.750650    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kpx6\" (UniqueName: \"kubernetes.io/projected/da806a2e-7fdf-4973-b36b-b646682281a4-kube-api-access-5kpx6\") pod \"metrics-server-86b96456cd-zvh4c\" (UID: \"da806a2e-7fdf-4973-b36b-b646682281a4\") " pod="openshift-monitoring/metrics-server-86b96456cd-zvh4c"
Sep 12 00:34:09.750758 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:09.750700    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da806a2e-7fdf-4973-b36b-b646682281a4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-86b96456cd-zvh4c\" (UID: \"da806a2e-7fdf-4973-b36b-b646682281a4\") " pod="openshift-monitoring/metrics-server-86b96456cd-zvh4c"
Sep 12 00:34:09.750758 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:09.750714    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/da806a2e-7fdf-4973-b36b-b646682281a4-client-ca-bundle\") pod \"metrics-server-86b96456cd-zvh4c\" (UID: \"da806a2e-7fdf-4973-b36b-b646682281a4\") " pod="openshift-monitoring/metrics-server-86b96456cd-zvh4c"
Sep 12 00:34:09.751567 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:09.751530    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/da806a2e-7fdf-4973-b36b-b646682281a4-audit-log\") pod \"metrics-server-86b96456cd-zvh4c\" (UID: \"da806a2e-7fdf-4973-b36b-b646682281a4\") " pod="openshift-monitoring/metrics-server-86b96456cd-zvh4c"
Sep 12 00:34:09.752224 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:09.752153    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/da806a2e-7fdf-4973-b36b-b646682281a4-metrics-server-audit-profiles\") pod \"metrics-server-86b96456cd-zvh4c\" (UID: \"da806a2e-7fdf-4973-b36b-b646682281a4\") " pod="openshift-monitoring/metrics-server-86b96456cd-zvh4c"
Sep 12 00:34:09.752861 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:09.752823    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da806a2e-7fdf-4973-b36b-b646682281a4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-86b96456cd-zvh4c\" (UID: \"da806a2e-7fdf-4973-b36b-b646682281a4\") " pod="openshift-monitoring/metrics-server-86b96456cd-zvh4c"
Sep 12 00:34:09.754796 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:09.754771    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/da806a2e-7fdf-4973-b36b-b646682281a4-secret-metrics-server-client-certs\") pod \"metrics-server-86b96456cd-zvh4c\" (UID: \"da806a2e-7fdf-4973-b36b-b646682281a4\") " pod="openshift-monitoring/metrics-server-86b96456cd-zvh4c"
Sep 12 00:34:09.754918 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:09.754898    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/da806a2e-7fdf-4973-b36b-b646682281a4-client-ca-bundle\") pod \"metrics-server-86b96456cd-zvh4c\" (UID: \"da806a2e-7fdf-4973-b36b-b646682281a4\") " pod="openshift-monitoring/metrics-server-86b96456cd-zvh4c"
Sep 12 00:34:09.755521 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:09.755470    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/da806a2e-7fdf-4973-b36b-b646682281a4-secret-metrics-server-tls\") pod \"metrics-server-86b96456cd-zvh4c\" (UID: \"da806a2e-7fdf-4973-b36b-b646682281a4\") " pod="openshift-monitoring/metrics-server-86b96456cd-zvh4c"
Sep 12 00:34:09.759203 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:09.759179    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kpx6\" (UniqueName: \"kubernetes.io/projected/da806a2e-7fdf-4973-b36b-b646682281a4-kube-api-access-5kpx6\") pod \"metrics-server-86b96456cd-zvh4c\" (UID: \"da806a2e-7fdf-4973-b36b-b646682281a4\") " pod="openshift-monitoring/metrics-server-86b96456cd-zvh4c"
Sep 12 00:34:09.805125 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:09.805078    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-86b96456cd-zvh4c"
Sep 12 00:34:09.852561 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:09.852519    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:11.275753 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.275711    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 12 00:34:11.280540 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.280493    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.282991 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.282966    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 12 00:34:11.283459 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.283439    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 12 00:34:11.283701 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.283683    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 12 00:34:11.284178 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.283949    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-6djp5\""
Sep 12 00:34:11.284475 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.284456    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 12 00:34:11.285163 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.284752    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 12 00:34:11.285163 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.284764    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 12 00:34:11.285163 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.284780    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 12 00:34:11.285163 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.284804    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 12 00:34:11.285163 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.284815    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 12 00:34:11.285163 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.284819    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 12 00:34:11.285163 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.284877    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-etefrcrd49g14\""
Sep 12 00:34:11.285163 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.285051    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 12 00:34:11.285163 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.285108    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 12 00:34:11.289606 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.289582    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 12 00:34:11.292618 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.292328    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 12 00:34:11.298346 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.298320    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 12 00:34:11.337476 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.337405    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:11.367120 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.366305    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.367120 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.366357    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bp2bb\" (UniqueName: \"kubernetes.io/projected/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-kube-api-access-bp2bb\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.367120 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.366401    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.367120 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.366424    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-web-config\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.367120 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.366452    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.367120 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.366497    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.367120 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.366524    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.367120 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.366561    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.367120 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.366585    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-config\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.367120 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.366606    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-config-out\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.367120 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.366631    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.367120 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.366665    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.367120 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.366689    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.367120 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.366715    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.367120 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.366739    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.367120 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.366762    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.367966 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.366788    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.367966 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.366830    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.367966 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.366860    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.367966 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.366894    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.428850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.428816    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:11.469608 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.469567    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.469784 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.469624    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.469784 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.469663    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.469784 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.469693    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-config\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.469784 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.469718    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-config-out\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.469784 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.469746    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.469784 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.469779    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.470114 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.469806    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.470114 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.469835    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.470114 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.469864    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.470114 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.469891    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.470114 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.469919    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.470114 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.469961    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.470114 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.469993    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.470114 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.470026    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.470114 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.470058    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.470114 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.470108    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bp2bb\" (UniqueName: \"kubernetes.io/projected/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-kube-api-access-bp2bb\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.470593 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.470148    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.470593 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.470162    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-web-config\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.470593 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.470179    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.470748 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.470709    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.471901 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.471279    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.472597 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.472574    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.474629 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.473716    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.474629 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.474129    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.474629 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.474441    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.474629 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.474584    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.475187 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.475144    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.475565 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.475527    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.476231 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.476210    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.476339 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.476305    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.476598 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.476578    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-config-out\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.476691 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.476593    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.477115 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.477072    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.478229 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.477723    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.478229 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.478194    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-config\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.478810 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.478767    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-web-config\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.479708 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.479667    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.480420 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.480379    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bp2bb\" (UniqueName: \"kubernetes.io/projected/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-kube-api-access-bp2bb\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.481063 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.481038    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:11.526374 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.526293    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:11.597701 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:11.597661    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:12.014521 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:12.014490    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:12.066827 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:12.066781    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:16.023465 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:16.023425    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:16.123436 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:16.123383    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:16.219920 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:16.219877    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:17.549418 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:17.549384    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:17.650474 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:17.650442    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:17.732777 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:34:17.732732    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-7tkvx" podUID="746054e5-684e-4c55-824b-6fdde562eb02"
Sep 12 00:34:17.749743 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:17.749657    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:18.010606 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:18.010534    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:18.108813 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:18.108747    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:18.283761 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:18.283446    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-86b96456cd-zvh4c"]
Sep 12 00:34:18.300118 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:18.300074    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 12 00:34:18.302507 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:34:18.302478    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99f6922b_0b37_4b49_b685_bbfcbb0d8cee.slice/crio-ff3876edd86d6ce8c2f9ab6b242789b6b5cf652036e575cb0ba1babf83010191 WatchSource:0}: Error finding container ff3876edd86d6ce8c2f9ab6b242789b6b5cf652036e575cb0ba1babf83010191: Status 404 returned error can't find the container with id ff3876edd86d6ce8c2f9ab6b242789b6b5cf652036e575cb0ba1babf83010191
Sep 12 00:34:18.661638 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:18.661404    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd" event={"ID":"5f89c9a5-1e8c-4e54-be92-c104eb70007f","Type":"ContainerStarted","Data":"15629c3c0fef7c59b18029440f84d5aedb9b69edb19a3de2bc7f2f6c57acc6cc"}
Sep 12 00:34:18.661638 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:18.661501    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd" event={"ID":"5f89c9a5-1e8c-4e54-be92-c104eb70007f","Type":"ContainerStarted","Data":"333c2d55536cd9e58a73f4718530421903313d5209b408cd5f57436f89a935f3"}
Sep 12 00:34:18.661638 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:18.661541    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd" event={"ID":"5f89c9a5-1e8c-4e54-be92-c104eb70007f","Type":"ContainerStarted","Data":"34cf5a473798707775624073fbe15194576da17e0e82a49fd8d2c21f3cf4b788"}
Sep 12 00:34:18.663181 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:18.663152    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-55c4s" event={"ID":"df843273-bc32-44d4-95bd-0ed4b1db206c","Type":"ContainerStarted","Data":"c23b4af56add3fa5c3bc9959bb256b720517722b48a01767b35cada5b4eebbf0"}
Sep 12 00:34:18.663608 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:18.663474    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-55c4s"
Sep 12 00:34:18.665439 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:18.665062    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"99f6922b-0b37-4b49-b685-bbfcbb0d8cee","Type":"ContainerStarted","Data":"ff3876edd86d6ce8c2f9ab6b242789b6b5cf652036e575cb0ba1babf83010191"}
Sep 12 00:34:18.667347 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:18.667326    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66665b5885-wbfxh" event={"ID":"3bba313c-7ee4-46de-b7bf-47e15ebb1add","Type":"ContainerStarted","Data":"544e3dc6639bb0f4cf41895a9b6be39668be9e7df83a0d1cacda5ad74cbd1a54"}
Sep 12 00:34:18.669117 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:18.669076    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-86b96456cd-zvh4c" event={"ID":"da806a2e-7fdf-4973-b36b-b646682281a4","Type":"ContainerStarted","Data":"a57d5ccbfc30b277632c407ab2da6cb2dc94cfc4e0480b7b4a6089c76e7d37da"}
Sep 12 00:34:18.673478 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:18.673458    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-55c4s"
Sep 12 00:34:18.685448 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:18.685307    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-55c4s" podStartSLOduration=2.046256795 podStartE2EDuration="21.685289986s" totalImagesPullingTime="19.639033191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:33:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:34:18.684141635 +0000 UTC m=+295.358200494" watchObservedRunningTime="2026-09-12 00:34:18.685289986 +0000 UTC m=+295.359348848"
Sep 12 00:34:18.727290 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:18.726839    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66665b5885-wbfxh" podStartSLOduration=2.3183956009999998 podStartE2EDuration="12.726816022s" totalImagesPullingTime="10.408420421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:34:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:34:18.72521233 +0000 UTC m=+295.399271190" watchObservedRunningTime="2026-09-12 00:34:18.726816022 +0000 UTC m=+295.400874882"
Sep 12 00:34:19.679148 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:19.678919    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5wzpq" event={"ID":"f0b419ed-e351-402a-9b9b-ff1534048f68","Type":"ContainerStarted","Data":"59ee0c5c0b17c30ac68ec39202b80874734626f8ca5586d99293d760fdcf6814"}
Sep 12 00:34:19.679148 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:19.678986    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5wzpq" event={"ID":"f0b419ed-e351-402a-9b9b-ff1534048f68","Type":"ContainerStarted","Data":"455e88679afccd60d2b0cb28cfa5b7905fd2ac470e568b9271c59522e62a4347"}
Sep 12 00:34:19.679660 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:19.679477    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5wzpq"
Sep 12 00:34:19.703556 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:19.703502    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:19.798057 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:19.797970    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:20.372628 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:20.372589    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:20.473537 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:20.473497    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:21.690113 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:21.690063    2558 generic.go:356] "Generic (PLEG): container finished" podID="99f6922b-0b37-4b49-b685-bbfcbb0d8cee" containerID="82c775245a20837afd85a773cb45f3b1e95f6b286347ac10e2c7d5af3fb4f9c1" exitCode=0
Sep 12 00:34:21.690496 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:21.690127    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"99f6922b-0b37-4b49-b685-bbfcbb0d8cee","Type":"ContainerDied","Data":"82c775245a20837afd85a773cb45f3b1e95f6b286347ac10e2c7d5af3fb4f9c1"}
Sep 12 00:34:21.692101 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:21.692058    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-86b96456cd-zvh4c" event={"ID":"da806a2e-7fdf-4973-b36b-b646682281a4","Type":"ContainerStarted","Data":"d58a8918e5d1e2f56f1ebbe56fec0a79ff15d08856e2f772e4a05a3c61f385e6"}
Sep 12 00:34:21.697692 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:21.697531    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd" event={"ID":"5f89c9a5-1e8c-4e54-be92-c104eb70007f","Type":"ContainerStarted","Data":"e25b35b70415224dd40069d7d6dbbca663a07072176ca26ceca909fa7bda346c"}
Sep 12 00:34:21.697692 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:21.697592    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd" event={"ID":"5f89c9a5-1e8c-4e54-be92-c104eb70007f","Type":"ContainerStarted","Data":"ea55c8d97db7d5e6d130a5ce29757e3f82597b139384d751f39c559b2f197992"}
Sep 12 00:34:21.717897 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:21.717841    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5wzpq" podStartSLOduration=253.360975813 podStartE2EDuration="4m23.717821304s" totalImagesPullingTime="10.356845491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:29:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:34:19.702995093 +0000 UTC m=+296.377053953" watchObservedRunningTime="2026-09-12 00:34:21.717821304 +0000 UTC m=+298.391880164"
Sep 12 00:34:21.732518 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:21.732467    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-86b96456cd-zvh4c" podStartSLOduration=9.677745261 podStartE2EDuration="12.732445857s" totalImagesPullingTime="3.054700596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:34:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:34:21.731282064 +0000 UTC m=+298.405340928" watchObservedRunningTime="2026-09-12 00:34:21.732445857 +0000 UTC m=+298.406504718"
Sep 12 00:34:22.709184 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:22.709138    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd" event={"ID":"5f89c9a5-1e8c-4e54-be92-c104eb70007f","Type":"ContainerStarted","Data":"26d27444d3baf86489e39bbf5d15e0972c9cbfbbd009efc2df3f0d93c9ef449d"}
Sep 12 00:34:22.709688 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:22.709624    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd"
Sep 12 00:34:22.732490 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:22.732192    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd" podStartSLOduration=2.310350247 podStartE2EDuration="15.732168721s" totalImagesPullingTime="13.421818474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:34:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:34:22.731580153 +0000 UTC m=+299.405639014" watchObservedRunningTime="2026-09-12 00:34:22.732168721 +0000 UTC m=+299.406227580"
Sep 12 00:34:23.680990 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:23.680954    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-czf8h_dd90996a-5428-4ccf-a154-c0aa8def9789/console-operator/1.log"
Sep 12 00:34:23.681485 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:23.681464    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-czf8h_dd90996a-5428-4ccf-a154-c0aa8def9789/console-operator/1.log"
Sep 12 00:34:23.700460 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:23.700238    2558 kubelet.go:1648] "Image garbage collection succeeded"
Sep 12 00:34:23.724921 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:23.724889    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-84d9f6d8bf-grjpd"
Sep 12 00:34:25.199056 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:25.198832    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:25.254019 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:25.253978    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:26.505762 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:26.505692    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:26.558275 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:26.558245    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:26.737872 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:26.737841    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"99f6922b-0b37-4b49-b685-bbfcbb0d8cee","Type":"ContainerStarted","Data":"6d98ece9cbf38af7b63ccea3195a72cfd411605794868d09df8c5b3accfe7e62"}
Sep 12 00:34:26.738049 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:26.737921    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"99f6922b-0b37-4b49-b685-bbfcbb0d8cee","Type":"ContainerStarted","Data":"8df5c8190048cc46635b1cf2832d1a6d60352229b0dd747ecb55b599f2c9c6d5"}
Sep 12 00:34:26.738049 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:26.737960    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"99f6922b-0b37-4b49-b685-bbfcbb0d8cee","Type":"ContainerStarted","Data":"f93541d0e28214c241b40444ed13f7939b6c687a7fc45a2c9ae7388292352783"}
Sep 12 00:34:26.738049 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:26.737998    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"99f6922b-0b37-4b49-b685-bbfcbb0d8cee","Type":"ContainerStarted","Data":"505e99ca541360ee2b869790671261b183b3f19ee25c35ee1b7f08ffd39314b5"}
Sep 12 00:34:26.738049 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:26.738031    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"99f6922b-0b37-4b49-b685-bbfcbb0d8cee","Type":"ContainerStarted","Data":"0e7eb89534ff6193d60743940d187f8a0ef6c0c4c93b2151e610bd7a6f930a9c"}
Sep 12 00:34:26.738237 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:26.738064    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"99f6922b-0b37-4b49-b685-bbfcbb0d8cee","Type":"ContainerStarted","Data":"913ff5932cb76084ab97e259839a13538f8f3bc6ebfabdc73526832ade7891bf"}
Sep 12 00:34:26.765423 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:26.765366    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=8.512274815 podStartE2EDuration="15.765350049s" totalImagesPullingTime="7.2235402s" totalInitContainerRuntime="29.535034ms" isStatefulPod=false podCreationTimestamp="2026-09-12 00:34:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:34:26.763308996 +0000 UTC m=+303.437367848" watchObservedRunningTime="2026-09-12 00:34:26.765350049 +0000 UTC m=+303.439408977"
Sep 12 00:34:27.310220 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:27.310180    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66665b5885-wbfxh"
Sep 12 00:34:27.310483 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:27.310266    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66665b5885-wbfxh"
Sep 12 00:34:27.316132 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:27.316080    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66665b5885-wbfxh"
Sep 12 00:34:27.528689 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:27.528657    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66665b5885-wbfxh"]
Sep 12 00:34:27.649450 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:27.649345    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:27.747693 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:27.747669    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66665b5885-wbfxh"
Sep 12 00:34:28.641724 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:28.641688    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:28.694131 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:28.694105    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:28.731701 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:28.731681    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:34:29.755516 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:29.755483    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:29.804939 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:29.804896    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:29.805687 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:29.805672    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-86b96456cd-zvh4c"
Sep 12 00:34:29.805761 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:29.805744    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-86b96456cd-zvh4c"
Sep 12 00:34:30.699691 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:30.699661    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5wzpq"
Sep 12 00:34:31.597978 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:31.597940    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:34:34.441328 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:34.441295    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:34.543097 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:34.543050    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:40.201304 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:40.201273    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:40.249914 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:40.249881    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:43.446232 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:43.446193    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:34:43.809283 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:43.809177    2558 generic.go:356] "Generic (PLEG): container finished" podID="3a305be2-8e7b-460c-adbf-691ae6c79683" containerID="0a11e6ad6d329b5038b7292a3bf3eaffa7fc110ee132657b71b403ab70eabf6f" exitCode=0
Sep 12 00:34:43.809446 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:43.809323    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-6wxf6" event={"ID":"3a305be2-8e7b-460c-adbf-691ae6c79683","Type":"ContainerDied","Data":"0a11e6ad6d329b5038b7292a3bf3eaffa7fc110ee132657b71b403ab70eabf6f"}
Sep 12 00:34:43.809802 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:43.809780    2558 scope.go:117] "RemoveContainer" containerID="0a11e6ad6d329b5038b7292a3bf3eaffa7fc110ee132657b71b403ab70eabf6f"
Sep 12 00:34:44.815162 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:44.815129    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-6wxf6" event={"ID":"3a305be2-8e7b-460c-adbf-691ae6c79683","Type":"ContainerStarted","Data":"abc1ef934cb64e380e694df7ddfbb835b7dd7a78b57d0a6e42d823b4bf381154"}
Sep 12 00:34:49.811320 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:49.811296    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-86b96456cd-zvh4c"
Sep 12 00:34:49.815063 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:49.815043    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-86b96456cd-zvh4c"
Sep 12 00:34:53.767104 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:53.767047    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-66665b5885-wbfxh" podUID="3bba313c-7ee4-46de-b7bf-47e15ebb1add" containerName="console" containerID="cri-o://544e3dc6639bb0f4cf41895a9b6be39668be9e7df83a0d1cacda5ad74cbd1a54" gracePeriod=15
Sep 12 00:34:54.105815 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:54.105787    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66665b5885-wbfxh"
Sep 12 00:34:54.270553 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:54.270519    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3bba313c-7ee4-46de-b7bf-47e15ebb1add-console-oauth-config\") pod \"3bba313c-7ee4-46de-b7bf-47e15ebb1add\" (UID: \"3bba313c-7ee4-46de-b7bf-47e15ebb1add\") "
Sep 12 00:34:54.270749 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:54.270587    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3bba313c-7ee4-46de-b7bf-47e15ebb1add-console-config\") pod \"3bba313c-7ee4-46de-b7bf-47e15ebb1add\" (UID: \"3bba313c-7ee4-46de-b7bf-47e15ebb1add\") "
Sep 12 00:34:54.270749 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:54.270617    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3bba313c-7ee4-46de-b7bf-47e15ebb1add-oauth-serving-cert\") pod \"3bba313c-7ee4-46de-b7bf-47e15ebb1add\" (UID: \"3bba313c-7ee4-46de-b7bf-47e15ebb1add\") "
Sep 12 00:34:54.270749 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:54.270663    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3bba313c-7ee4-46de-b7bf-47e15ebb1add-service-ca\") pod \"3bba313c-7ee4-46de-b7bf-47e15ebb1add\" (UID: \"3bba313c-7ee4-46de-b7bf-47e15ebb1add\") "
Sep 12 00:34:54.270749 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:54.270684    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3bba313c-7ee4-46de-b7bf-47e15ebb1add-console-serving-cert\") pod \"3bba313c-7ee4-46de-b7bf-47e15ebb1add\" (UID: \"3bba313c-7ee4-46de-b7bf-47e15ebb1add\") "
Sep 12 00:34:54.270749 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:54.270718    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-btnr4\" (UniqueName: \"kubernetes.io/projected/3bba313c-7ee4-46de-b7bf-47e15ebb1add-kube-api-access-btnr4\") pod \"3bba313c-7ee4-46de-b7bf-47e15ebb1add\" (UID: \"3bba313c-7ee4-46de-b7bf-47e15ebb1add\") "
Sep 12 00:34:54.271101 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:54.271053    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bba313c-7ee4-46de-b7bf-47e15ebb1add-service-ca" (OuterVolumeSpecName: "service-ca") pod "3bba313c-7ee4-46de-b7bf-47e15ebb1add" (UID: "3bba313c-7ee4-46de-b7bf-47e15ebb1add"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:34:54.271161 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:54.271063    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bba313c-7ee4-46de-b7bf-47e15ebb1add-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3bba313c-7ee4-46de-b7bf-47e15ebb1add" (UID: "3bba313c-7ee4-46de-b7bf-47e15ebb1add"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:34:54.271161 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:54.271060    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bba313c-7ee4-46de-b7bf-47e15ebb1add-console-config" (OuterVolumeSpecName: "console-config") pod "3bba313c-7ee4-46de-b7bf-47e15ebb1add" (UID: "3bba313c-7ee4-46de-b7bf-47e15ebb1add"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:34:54.273020 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:54.272989    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bba313c-7ee4-46de-b7bf-47e15ebb1add-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3bba313c-7ee4-46de-b7bf-47e15ebb1add" (UID: "3bba313c-7ee4-46de-b7bf-47e15ebb1add"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:34:54.273126 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:54.273025    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bba313c-7ee4-46de-b7bf-47e15ebb1add-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3bba313c-7ee4-46de-b7bf-47e15ebb1add" (UID: "3bba313c-7ee4-46de-b7bf-47e15ebb1add"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:34:54.273126 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:54.273035    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bba313c-7ee4-46de-b7bf-47e15ebb1add-kube-api-access-btnr4" (OuterVolumeSpecName: "kube-api-access-btnr4") pod "3bba313c-7ee4-46de-b7bf-47e15ebb1add" (UID: "3bba313c-7ee4-46de-b7bf-47e15ebb1add"). InnerVolumeSpecName "kube-api-access-btnr4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:34:54.373707 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:54.373624    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btnr4\" (UniqueName: \"kubernetes.io/projected/3bba313c-7ee4-46de-b7bf-47e15ebb1add-kube-api-access-btnr4\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:34:54.373707 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:54.373652    2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3bba313c-7ee4-46de-b7bf-47e15ebb1add-console-oauth-config\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:34:54.373707 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:54.373662    2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3bba313c-7ee4-46de-b7bf-47e15ebb1add-console-config\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:34:54.373707 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:54.373671    2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3bba313c-7ee4-46de-b7bf-47e15ebb1add-oauth-serving-cert\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:34:54.373707 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:54.373679    2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3bba313c-7ee4-46de-b7bf-47e15ebb1add-service-ca\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:34:54.373707 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:54.373687    2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3bba313c-7ee4-46de-b7bf-47e15ebb1add-console-serving-cert\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:34:54.856362 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:54.856323    2558 generic.go:356] "Generic (PLEG): container finished" podID="3bba313c-7ee4-46de-b7bf-47e15ebb1add" containerID="544e3dc6639bb0f4cf41895a9b6be39668be9e7df83a0d1cacda5ad74cbd1a54" exitCode=0
Sep 12 00:34:54.856802 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:54.856383    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66665b5885-wbfxh"
Sep 12 00:34:54.856802 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:54.856411    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66665b5885-wbfxh" event={"ID":"3bba313c-7ee4-46de-b7bf-47e15ebb1add","Type":"ContainerDied","Data":"544e3dc6639bb0f4cf41895a9b6be39668be9e7df83a0d1cacda5ad74cbd1a54"}
Sep 12 00:34:54.856802 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:54.856467    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66665b5885-wbfxh" event={"ID":"3bba313c-7ee4-46de-b7bf-47e15ebb1add","Type":"ContainerDied","Data":"91e18c6a1703eae6c199d16baba1425f8a436b400600078b836d2ef1f686a937"}
Sep 12 00:34:54.856802 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:54.856490    2558 scope.go:117] "RemoveContainer" containerID="544e3dc6639bb0f4cf41895a9b6be39668be9e7df83a0d1cacda5ad74cbd1a54"
Sep 12 00:34:54.858113 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:54.858064    2558 generic.go:356] "Generic (PLEG): container finished" podID="fc47c0e2-02fe-408b-b0a5-5b046a81f1d7" containerID="e8cc5985156bec469bc7d99aaa6346f8248554b5e90d9ac1a3210b97f798c925" exitCode=0
Sep 12 00:34:54.858307 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:54.858240    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-cnml8" event={"ID":"fc47c0e2-02fe-408b-b0a5-5b046a81f1d7","Type":"ContainerDied","Data":"e8cc5985156bec469bc7d99aaa6346f8248554b5e90d9ac1a3210b97f798c925"}
Sep 12 00:34:54.858719 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:54.858702    2558 scope.go:117] "RemoveContainer" containerID="e8cc5985156bec469bc7d99aaa6346f8248554b5e90d9ac1a3210b97f798c925"
Sep 12 00:34:54.872977 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:54.872959    2558 scope.go:117] "RemoveContainer" containerID="544e3dc6639bb0f4cf41895a9b6be39668be9e7df83a0d1cacda5ad74cbd1a54"
Sep 12 00:34:54.873324 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:34:54.873298    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"544e3dc6639bb0f4cf41895a9b6be39668be9e7df83a0d1cacda5ad74cbd1a54\": container with ID starting with 544e3dc6639bb0f4cf41895a9b6be39668be9e7df83a0d1cacda5ad74cbd1a54 not found: ID does not exist" containerID="544e3dc6639bb0f4cf41895a9b6be39668be9e7df83a0d1cacda5ad74cbd1a54"
Sep 12 00:34:54.873414 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:54.873337    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"544e3dc6639bb0f4cf41895a9b6be39668be9e7df83a0d1cacda5ad74cbd1a54"} err="failed to get container status \"544e3dc6639bb0f4cf41895a9b6be39668be9e7df83a0d1cacda5ad74cbd1a54\": rpc error: code = NotFound desc = could not find container \"544e3dc6639bb0f4cf41895a9b6be39668be9e7df83a0d1cacda5ad74cbd1a54\": container with ID starting with 544e3dc6639bb0f4cf41895a9b6be39668be9e7df83a0d1cacda5ad74cbd1a54 not found: ID does not exist"
Sep 12 00:34:54.890051 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:54.890023    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66665b5885-wbfxh"]
Sep 12 00:34:54.892461 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:54.892438    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-66665b5885-wbfxh"]
Sep 12 00:34:55.736930 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:55.736892    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3bba313c-7ee4-46de-b7bf-47e15ebb1add" path="/var/lib/kubelet/pods/3bba313c-7ee4-46de-b7bf-47e15ebb1add/volumes"
Sep 12 00:34:55.864814 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:34:55.864784    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-cnml8" event={"ID":"fc47c0e2-02fe-408b-b0a5-5b046a81f1d7","Type":"ContainerStarted","Data":"95e02643b53f7418a3d278198040b5961d7a194fe4ecea64ccb1803298774b18"}
Sep 12 00:35:11.598260 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:11.598227    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:11.614785 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:11.614757    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:11.944277 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:11.944251    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:22.808990 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:22.808898    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:35:22.861239 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:22.861205    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:35:24.236327 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:24.236292    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:35:24.705028 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:24.704992    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:35:29.601508 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:29.601473    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 12 00:35:29.602014 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:29.601764    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="99f6922b-0b37-4b49-b685-bbfcbb0d8cee" containerName="prometheus" containerID="cri-o://913ff5932cb76084ab97e259839a13538f8f3bc6ebfabdc73526832ade7891bf" gracePeriod=600
Sep 12 00:35:29.602014 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:29.601806    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="99f6922b-0b37-4b49-b685-bbfcbb0d8cee" containerName="kube-rbac-proxy" containerID="cri-o://8df5c8190048cc46635b1cf2832d1a6d60352229b0dd747ecb55b599f2c9c6d5" gracePeriod=600
Sep 12 00:35:29.602014 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:29.601832    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="99f6922b-0b37-4b49-b685-bbfcbb0d8cee" containerName="thanos-sidecar" containerID="cri-o://505e99ca541360ee2b869790671261b183b3f19ee25c35ee1b7f08ffd39314b5" gracePeriod=600
Sep 12 00:35:29.602014 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:29.601853    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="99f6922b-0b37-4b49-b685-bbfcbb0d8cee" containerName="kube-rbac-proxy-thanos" containerID="cri-o://6d98ece9cbf38af7b63ccea3195a72cfd411605794868d09df8c5b3accfe7e62" gracePeriod=600
Sep 12 00:35:29.602014 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:29.601834    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="99f6922b-0b37-4b49-b685-bbfcbb0d8cee" containerName="kube-rbac-proxy-web" containerID="cri-o://f93541d0e28214c241b40444ed13f7939b6c687a7fc45a2c9ae7388292352783" gracePeriod=600
Sep 12 00:35:29.602014 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:29.601996    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="99f6922b-0b37-4b49-b685-bbfcbb0d8cee" containerName="config-reloader" containerID="cri-o://0e7eb89534ff6193d60743940d187f8a0ef6c0c4c93b2151e610bd7a6f930a9c" gracePeriod=600
Sep 12 00:35:29.997879 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:29.997847    2558 generic.go:356] "Generic (PLEG): container finished" podID="99f6922b-0b37-4b49-b685-bbfcbb0d8cee" containerID="6d98ece9cbf38af7b63ccea3195a72cfd411605794868d09df8c5b3accfe7e62" exitCode=0
Sep 12 00:35:29.997879 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:29.997870    2558 generic.go:356] "Generic (PLEG): container finished" podID="99f6922b-0b37-4b49-b685-bbfcbb0d8cee" containerID="8df5c8190048cc46635b1cf2832d1a6d60352229b0dd747ecb55b599f2c9c6d5" exitCode=0
Sep 12 00:35:29.997879 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:29.997876    2558 generic.go:356] "Generic (PLEG): container finished" podID="99f6922b-0b37-4b49-b685-bbfcbb0d8cee" containerID="505e99ca541360ee2b869790671261b183b3f19ee25c35ee1b7f08ffd39314b5" exitCode=0
Sep 12 00:35:29.997879 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:29.997883    2558 generic.go:356] "Generic (PLEG): container finished" podID="99f6922b-0b37-4b49-b685-bbfcbb0d8cee" containerID="0e7eb89534ff6193d60743940d187f8a0ef6c0c4c93b2151e610bd7a6f930a9c" exitCode=0
Sep 12 00:35:29.997879 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:29.997888    2558 generic.go:356] "Generic (PLEG): container finished" podID="99f6922b-0b37-4b49-b685-bbfcbb0d8cee" containerID="913ff5932cb76084ab97e259839a13538f8f3bc6ebfabdc73526832ade7891bf" exitCode=0
Sep 12 00:35:29.998250 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:29.997917    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"99f6922b-0b37-4b49-b685-bbfcbb0d8cee","Type":"ContainerDied","Data":"6d98ece9cbf38af7b63ccea3195a72cfd411605794868d09df8c5b3accfe7e62"}
Sep 12 00:35:29.998250 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:29.997992    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"99f6922b-0b37-4b49-b685-bbfcbb0d8cee","Type":"ContainerDied","Data":"8df5c8190048cc46635b1cf2832d1a6d60352229b0dd747ecb55b599f2c9c6d5"}
Sep 12 00:35:29.998250 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:29.998030    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"99f6922b-0b37-4b49-b685-bbfcbb0d8cee","Type":"ContainerDied","Data":"505e99ca541360ee2b869790671261b183b3f19ee25c35ee1b7f08ffd39314b5"}
Sep 12 00:35:29.998250 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:29.998064    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"99f6922b-0b37-4b49-b685-bbfcbb0d8cee","Type":"ContainerDied","Data":"0e7eb89534ff6193d60743940d187f8a0ef6c0c4c93b2151e610bd7a6f930a9c"}
Sep 12 00:35:29.998250 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:29.998121    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"99f6922b-0b37-4b49-b685-bbfcbb0d8cee","Type":"ContainerDied","Data":"913ff5932cb76084ab97e259839a13538f8f3bc6ebfabdc73526832ade7891bf"}
Sep 12 00:35:30.850891 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:30.850869    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.002920 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.002835    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-configmap-serving-certs-ca-bundle\") pod \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") "
Sep 12 00:35:31.002920 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.002882    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bp2bb\" (UniqueName: \"kubernetes.io/projected/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-kube-api-access-bp2bb\") pod \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") "
Sep 12 00:35:31.002920 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.002906    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-metrics-client-certs\") pod \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") "
Sep 12 00:35:31.003200 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.002929    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") "
Sep 12 00:35:31.003200 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.002976    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-thanos-prometheus-http-client-file\") pod \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") "
Sep 12 00:35:31.003200 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.003017    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-tls-assets\") pod \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") "
Sep 12 00:35:31.003200 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.003041    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-prometheus-k8s-rulefiles-2\") pod \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") "
Sep 12 00:35:31.003200 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.003105    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-prometheus-k8s-rulefiles-0\") pod \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") "
Sep 12 00:35:31.003200 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.003136    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-prometheus-trusted-ca-bundle\") pod \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") "
Sep 12 00:35:31.003200 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.003159    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-prometheus-k8s-rulefiles-1\") pod \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") "
Sep 12 00:35:31.003200 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.003189    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-prometheus-k8s-db\") pod \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") "
Sep 12 00:35:31.003529 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.003218    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") "
Sep 12 00:35:31.003529 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.003245    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-configmap-kubelet-serving-ca-bundle\") pod \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") "
Sep 12 00:35:31.003529 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.003282    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-kube-rbac-proxy\") pod \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") "
Sep 12 00:35:31.003529 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.003285    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "99f6922b-0b37-4b49-b685-bbfcbb0d8cee" (UID: "99f6922b-0b37-4b49-b685-bbfcbb0d8cee"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:35:31.003529 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.003306    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-config-out\") pod \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") "
Sep 12 00:35:31.003529 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.003350    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-web-config\") pod \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") "
Sep 12 00:35:31.003912 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.003661    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "99f6922b-0b37-4b49-b685-bbfcbb0d8cee" (UID: "99f6922b-0b37-4b49-b685-bbfcbb0d8cee"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:35:31.004282 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.004109    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "99f6922b-0b37-4b49-b685-bbfcbb0d8cee" (UID: "99f6922b-0b37-4b49-b685-bbfcbb0d8cee"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:35:31.004282 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.003937    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-config\") pod \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") "
Sep 12 00:35:31.006212 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.004847    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "99f6922b-0b37-4b49-b685-bbfcbb0d8cee" (UID: "99f6922b-0b37-4b49-b685-bbfcbb0d8cee"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:35:31.006212 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.005587    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "99f6922b-0b37-4b49-b685-bbfcbb0d8cee" (UID: "99f6922b-0b37-4b49-b685-bbfcbb0d8cee"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:35:31.006212 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.005637    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-grpc-tls\") pod \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") "
Sep 12 00:35:31.006212 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.005668    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-prometheus-k8s-tls\") pod \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") "
Sep 12 00:35:31.006212 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.005698    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-configmap-metrics-client-ca\") pod \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\" (UID: \"99f6922b-0b37-4b49-b685-bbfcbb0d8cee\") "
Sep 12 00:35:31.006212 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.005810    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "99f6922b-0b37-4b49-b685-bbfcbb0d8cee" (UID: "99f6922b-0b37-4b49-b685-bbfcbb0d8cee"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:35:31.006212 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.005999    2558 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:35:31.006212 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.006016    2558 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:35:31.006212 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.006031    2558 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-prometheus-trusted-ca-bundle\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:35:31.006212 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.006046    2558 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:35:31.006212 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.006062    2558 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:35:31.006212 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.006076    2558 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:35:31.006866 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.006463    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "99f6922b-0b37-4b49-b685-bbfcbb0d8cee" (UID: "99f6922b-0b37-4b49-b685-bbfcbb0d8cee"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:35:31.007179 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.007101    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "99f6922b-0b37-4b49-b685-bbfcbb0d8cee" (UID: "99f6922b-0b37-4b49-b685-bbfcbb0d8cee"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:35:31.007805 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.007440    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-kube-api-access-bp2bb" (OuterVolumeSpecName: "kube-api-access-bp2bb") pod "99f6922b-0b37-4b49-b685-bbfcbb0d8cee" (UID: "99f6922b-0b37-4b49-b685-bbfcbb0d8cee"). InnerVolumeSpecName "kube-api-access-bp2bb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:35:31.007805 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.007510    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "99f6922b-0b37-4b49-b685-bbfcbb0d8cee" (UID: "99f6922b-0b37-4b49-b685-bbfcbb0d8cee"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:35:31.008127 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.008096    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "99f6922b-0b37-4b49-b685-bbfcbb0d8cee" (UID: "99f6922b-0b37-4b49-b685-bbfcbb0d8cee"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:35:31.008270 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.008245    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "99f6922b-0b37-4b49-b685-bbfcbb0d8cee" (UID: "99f6922b-0b37-4b49-b685-bbfcbb0d8cee"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:35:31.008536 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.008457    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "99f6922b-0b37-4b49-b685-bbfcbb0d8cee" (UID: "99f6922b-0b37-4b49-b685-bbfcbb0d8cee"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:35:31.008536 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.008519    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-config-out" (OuterVolumeSpecName: "config-out") pod "99f6922b-0b37-4b49-b685-bbfcbb0d8cee" (UID: "99f6922b-0b37-4b49-b685-bbfcbb0d8cee"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:35:31.009152 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.009125    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-config" (OuterVolumeSpecName: "config") pod "99f6922b-0b37-4b49-b685-bbfcbb0d8cee" (UID: "99f6922b-0b37-4b49-b685-bbfcbb0d8cee"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:35:31.009303 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.009270    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "99f6922b-0b37-4b49-b685-bbfcbb0d8cee" (UID: "99f6922b-0b37-4b49-b685-bbfcbb0d8cee"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:35:31.009601 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.009566    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "99f6922b-0b37-4b49-b685-bbfcbb0d8cee" (UID: "99f6922b-0b37-4b49-b685-bbfcbb0d8cee"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:35:31.009686 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.009653    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "99f6922b-0b37-4b49-b685-bbfcbb0d8cee" (UID: "99f6922b-0b37-4b49-b685-bbfcbb0d8cee"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:35:31.009739 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.009678    2558 generic.go:356] "Generic (PLEG): container finished" podID="99f6922b-0b37-4b49-b685-bbfcbb0d8cee" containerID="f93541d0e28214c241b40444ed13f7939b6c687a7fc45a2c9ae7388292352783" exitCode=0
Sep 12 00:35:31.009739 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.009731    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"99f6922b-0b37-4b49-b685-bbfcbb0d8cee","Type":"ContainerDied","Data":"f93541d0e28214c241b40444ed13f7939b6c687a7fc45a2c9ae7388292352783"}
Sep 12 00:35:31.009844 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.009780    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.009844 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.009804    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"99f6922b-0b37-4b49-b685-bbfcbb0d8cee","Type":"ContainerDied","Data":"ff3876edd86d6ce8c2f9ab6b242789b6b5cf652036e575cb0ba1babf83010191"}
Sep 12 00:35:31.009936 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.009853    2558 scope.go:117] "RemoveContainer" containerID="6d98ece9cbf38af7b63ccea3195a72cfd411605794868d09df8c5b3accfe7e62"
Sep 12 00:35:31.010037 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.010013    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "99f6922b-0b37-4b49-b685-bbfcbb0d8cee" (UID: "99f6922b-0b37-4b49-b685-bbfcbb0d8cee"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:35:31.020057 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.020030    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-web-config" (OuterVolumeSpecName: "web-config") pod "99f6922b-0b37-4b49-b685-bbfcbb0d8cee" (UID: "99f6922b-0b37-4b49-b685-bbfcbb0d8cee"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:35:31.026000 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.025982    2558 scope.go:117] "RemoveContainer" containerID="8df5c8190048cc46635b1cf2832d1a6d60352229b0dd747ecb55b599f2c9c6d5"
Sep 12 00:35:31.032976 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.032962    2558 scope.go:117] "RemoveContainer" containerID="f93541d0e28214c241b40444ed13f7939b6c687a7fc45a2c9ae7388292352783"
Sep 12 00:35:31.039385 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.039369    2558 scope.go:117] "RemoveContainer" containerID="505e99ca541360ee2b869790671261b183b3f19ee25c35ee1b7f08ffd39314b5"
Sep 12 00:35:31.045625 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.045604    2558 scope.go:117] "RemoveContainer" containerID="0e7eb89534ff6193d60743940d187f8a0ef6c0c4c93b2151e610bd7a6f930a9c"
Sep 12 00:35:31.051986 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.051969    2558 scope.go:117] "RemoveContainer" containerID="913ff5932cb76084ab97e259839a13538f8f3bc6ebfabdc73526832ade7891bf"
Sep 12 00:35:31.058806 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.058790    2558 scope.go:117] "RemoveContainer" containerID="82c775245a20837afd85a773cb45f3b1e95f6b286347ac10e2c7d5af3fb4f9c1"
Sep 12 00:35:31.065152 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.065134    2558 scope.go:117] "RemoveContainer" containerID="6d98ece9cbf38af7b63ccea3195a72cfd411605794868d09df8c5b3accfe7e62"
Sep 12 00:35:31.065393 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:35:31.065375    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d98ece9cbf38af7b63ccea3195a72cfd411605794868d09df8c5b3accfe7e62\": container with ID starting with 6d98ece9cbf38af7b63ccea3195a72cfd411605794868d09df8c5b3accfe7e62 not found: ID does not exist" containerID="6d98ece9cbf38af7b63ccea3195a72cfd411605794868d09df8c5b3accfe7e62"
Sep 12 00:35:31.065450 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.065404    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d98ece9cbf38af7b63ccea3195a72cfd411605794868d09df8c5b3accfe7e62"} err="failed to get container status \"6d98ece9cbf38af7b63ccea3195a72cfd411605794868d09df8c5b3accfe7e62\": rpc error: code = NotFound desc = could not find container \"6d98ece9cbf38af7b63ccea3195a72cfd411605794868d09df8c5b3accfe7e62\": container with ID starting with 6d98ece9cbf38af7b63ccea3195a72cfd411605794868d09df8c5b3accfe7e62 not found: ID does not exist"
Sep 12 00:35:31.065450 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.065423    2558 scope.go:117] "RemoveContainer" containerID="8df5c8190048cc46635b1cf2832d1a6d60352229b0dd747ecb55b599f2c9c6d5"
Sep 12 00:35:31.065662 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:35:31.065647    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8df5c8190048cc46635b1cf2832d1a6d60352229b0dd747ecb55b599f2c9c6d5\": container with ID starting with 8df5c8190048cc46635b1cf2832d1a6d60352229b0dd747ecb55b599f2c9c6d5 not found: ID does not exist" containerID="8df5c8190048cc46635b1cf2832d1a6d60352229b0dd747ecb55b599f2c9c6d5"
Sep 12 00:35:31.065700 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.065668    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8df5c8190048cc46635b1cf2832d1a6d60352229b0dd747ecb55b599f2c9c6d5"} err="failed to get container status \"8df5c8190048cc46635b1cf2832d1a6d60352229b0dd747ecb55b599f2c9c6d5\": rpc error: code = NotFound desc = could not find container \"8df5c8190048cc46635b1cf2832d1a6d60352229b0dd747ecb55b599f2c9c6d5\": container with ID starting with 8df5c8190048cc46635b1cf2832d1a6d60352229b0dd747ecb55b599f2c9c6d5 not found: ID does not exist"
Sep 12 00:35:31.065700 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.065690    2558 scope.go:117] "RemoveContainer" containerID="f93541d0e28214c241b40444ed13f7939b6c687a7fc45a2c9ae7388292352783"
Sep 12 00:35:31.065935 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:35:31.065918    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f93541d0e28214c241b40444ed13f7939b6c687a7fc45a2c9ae7388292352783\": container with ID starting with f93541d0e28214c241b40444ed13f7939b6c687a7fc45a2c9ae7388292352783 not found: ID does not exist" containerID="f93541d0e28214c241b40444ed13f7939b6c687a7fc45a2c9ae7388292352783"
Sep 12 00:35:31.065985 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.065941    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f93541d0e28214c241b40444ed13f7939b6c687a7fc45a2c9ae7388292352783"} err="failed to get container status \"f93541d0e28214c241b40444ed13f7939b6c687a7fc45a2c9ae7388292352783\": rpc error: code = NotFound desc = could not find container \"f93541d0e28214c241b40444ed13f7939b6c687a7fc45a2c9ae7388292352783\": container with ID starting with f93541d0e28214c241b40444ed13f7939b6c687a7fc45a2c9ae7388292352783 not found: ID does not exist"
Sep 12 00:35:31.065985 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.065955    2558 scope.go:117] "RemoveContainer" containerID="505e99ca541360ee2b869790671261b183b3f19ee25c35ee1b7f08ffd39314b5"
Sep 12 00:35:31.066218 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:35:31.066204    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"505e99ca541360ee2b869790671261b183b3f19ee25c35ee1b7f08ffd39314b5\": container with ID starting with 505e99ca541360ee2b869790671261b183b3f19ee25c35ee1b7f08ffd39314b5 not found: ID does not exist" containerID="505e99ca541360ee2b869790671261b183b3f19ee25c35ee1b7f08ffd39314b5"
Sep 12 00:35:31.066255 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.066221    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"505e99ca541360ee2b869790671261b183b3f19ee25c35ee1b7f08ffd39314b5"} err="failed to get container status \"505e99ca541360ee2b869790671261b183b3f19ee25c35ee1b7f08ffd39314b5\": rpc error: code = NotFound desc = could not find container \"505e99ca541360ee2b869790671261b183b3f19ee25c35ee1b7f08ffd39314b5\": container with ID starting with 505e99ca541360ee2b869790671261b183b3f19ee25c35ee1b7f08ffd39314b5 not found: ID does not exist"
Sep 12 00:35:31.066255 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.066234    2558 scope.go:117] "RemoveContainer" containerID="0e7eb89534ff6193d60743940d187f8a0ef6c0c4c93b2151e610bd7a6f930a9c"
Sep 12 00:35:31.066451 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:35:31.066435    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e7eb89534ff6193d60743940d187f8a0ef6c0c4c93b2151e610bd7a6f930a9c\": container with ID starting with 0e7eb89534ff6193d60743940d187f8a0ef6c0c4c93b2151e610bd7a6f930a9c not found: ID does not exist" containerID="0e7eb89534ff6193d60743940d187f8a0ef6c0c4c93b2151e610bd7a6f930a9c"
Sep 12 00:35:31.066499 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.066456    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e7eb89534ff6193d60743940d187f8a0ef6c0c4c93b2151e610bd7a6f930a9c"} err="failed to get container status \"0e7eb89534ff6193d60743940d187f8a0ef6c0c4c93b2151e610bd7a6f930a9c\": rpc error: code = NotFound desc = could not find container \"0e7eb89534ff6193d60743940d187f8a0ef6c0c4c93b2151e610bd7a6f930a9c\": container with ID starting with 0e7eb89534ff6193d60743940d187f8a0ef6c0c4c93b2151e610bd7a6f930a9c not found: ID does not exist"
Sep 12 00:35:31.066499 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.066468    2558 scope.go:117] "RemoveContainer" containerID="913ff5932cb76084ab97e259839a13538f8f3bc6ebfabdc73526832ade7891bf"
Sep 12 00:35:31.066660 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:35:31.066642    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"913ff5932cb76084ab97e259839a13538f8f3bc6ebfabdc73526832ade7891bf\": container with ID starting with 913ff5932cb76084ab97e259839a13538f8f3bc6ebfabdc73526832ade7891bf not found: ID does not exist" containerID="913ff5932cb76084ab97e259839a13538f8f3bc6ebfabdc73526832ade7891bf"
Sep 12 00:35:31.066696 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.066664    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"913ff5932cb76084ab97e259839a13538f8f3bc6ebfabdc73526832ade7891bf"} err="failed to get container status \"913ff5932cb76084ab97e259839a13538f8f3bc6ebfabdc73526832ade7891bf\": rpc error: code = NotFound desc = could not find container \"913ff5932cb76084ab97e259839a13538f8f3bc6ebfabdc73526832ade7891bf\": container with ID starting with 913ff5932cb76084ab97e259839a13538f8f3bc6ebfabdc73526832ade7891bf not found: ID does not exist"
Sep 12 00:35:31.066696 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.066675    2558 scope.go:117] "RemoveContainer" containerID="82c775245a20837afd85a773cb45f3b1e95f6b286347ac10e2c7d5af3fb4f9c1"
Sep 12 00:35:31.066906 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:35:31.066890    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82c775245a20837afd85a773cb45f3b1e95f6b286347ac10e2c7d5af3fb4f9c1\": container with ID starting with 82c775245a20837afd85a773cb45f3b1e95f6b286347ac10e2c7d5af3fb4f9c1 not found: ID does not exist" containerID="82c775245a20837afd85a773cb45f3b1e95f6b286347ac10e2c7d5af3fb4f9c1"
Sep 12 00:35:31.066946 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.066908    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82c775245a20837afd85a773cb45f3b1e95f6b286347ac10e2c7d5af3fb4f9c1"} err="failed to get container status \"82c775245a20837afd85a773cb45f3b1e95f6b286347ac10e2c7d5af3fb4f9c1\": rpc error: code = NotFound desc = could not find container \"82c775245a20837afd85a773cb45f3b1e95f6b286347ac10e2c7d5af3fb4f9c1\": container with ID starting with 82c775245a20837afd85a773cb45f3b1e95f6b286347ac10e2c7d5af3fb4f9c1 not found: ID does not exist"
Sep 12 00:35:31.107503 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.107475    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bp2bb\" (UniqueName: \"kubernetes.io/projected/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-kube-api-access-bp2bb\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:35:31.107503 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.107501    2558 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-metrics-client-certs\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:35:31.107689 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.107513    2558 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:35:31.107689 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.107522    2558 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-thanos-prometheus-http-client-file\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:35:31.107689 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.107531    2558 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-tls-assets\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:35:31.107689 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.107539    2558 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-prometheus-k8s-db\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:35:31.107689 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.107547    2558 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:35:31.107689 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.107555    2558 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-kube-rbac-proxy\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:35:31.107689 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.107562    2558 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-config-out\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:35:31.107689 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.107579    2558 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-web-config\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:35:31.107689 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.107586    2558 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-config\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:35:31.107689 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.107593    2558 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-grpc-tls\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:35:31.107689 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.107600    2558 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-secret-prometheus-k8s-tls\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:35:31.107689 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.107628    2558 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99f6922b-0b37-4b49-b685-bbfcbb0d8cee-configmap-metrics-client-ca\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:35:31.333046 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.333012    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 12 00:35:31.337363 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.337335    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 12 00:35:31.360542 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.360514    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 12 00:35:31.360803 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.360791    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99f6922b-0b37-4b49-b685-bbfcbb0d8cee" containerName="kube-rbac-proxy-thanos"
Sep 12 00:35:31.360845 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.360805    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="99f6922b-0b37-4b49-b685-bbfcbb0d8cee" containerName="kube-rbac-proxy-thanos"
Sep 12 00:35:31.360845 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.360817    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99f6922b-0b37-4b49-b685-bbfcbb0d8cee" containerName="kube-rbac-proxy-web"
Sep 12 00:35:31.360845 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.360822    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="99f6922b-0b37-4b49-b685-bbfcbb0d8cee" containerName="kube-rbac-proxy-web"
Sep 12 00:35:31.360845 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.360829    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bba313c-7ee4-46de-b7bf-47e15ebb1add" containerName="console"
Sep 12 00:35:31.360845 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.360834    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bba313c-7ee4-46de-b7bf-47e15ebb1add" containerName="console"
Sep 12 00:35:31.360845 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.360841    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99f6922b-0b37-4b49-b685-bbfcbb0d8cee" containerName="config-reloader"
Sep 12 00:35:31.360845 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.360846    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="99f6922b-0b37-4b49-b685-bbfcbb0d8cee" containerName="config-reloader"
Sep 12 00:35:31.361034 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.360857    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99f6922b-0b37-4b49-b685-bbfcbb0d8cee" containerName="init-config-reloader"
Sep 12 00:35:31.361034 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.360862    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="99f6922b-0b37-4b49-b685-bbfcbb0d8cee" containerName="init-config-reloader"
Sep 12 00:35:31.361034 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.360869    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99f6922b-0b37-4b49-b685-bbfcbb0d8cee" containerName="thanos-sidecar"
Sep 12 00:35:31.361034 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.360874    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="99f6922b-0b37-4b49-b685-bbfcbb0d8cee" containerName="thanos-sidecar"
Sep 12 00:35:31.361034 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.360880    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99f6922b-0b37-4b49-b685-bbfcbb0d8cee" containerName="kube-rbac-proxy"
Sep 12 00:35:31.361034 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.360884    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="99f6922b-0b37-4b49-b685-bbfcbb0d8cee" containerName="kube-rbac-proxy"
Sep 12 00:35:31.361034 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.360890    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99f6922b-0b37-4b49-b685-bbfcbb0d8cee" containerName="prometheus"
Sep 12 00:35:31.361034 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.360895    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="99f6922b-0b37-4b49-b685-bbfcbb0d8cee" containerName="prometheus"
Sep 12 00:35:31.361034 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.360931    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="99f6922b-0b37-4b49-b685-bbfcbb0d8cee" containerName="kube-rbac-proxy-thanos"
Sep 12 00:35:31.361034 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.360940    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="99f6922b-0b37-4b49-b685-bbfcbb0d8cee" containerName="kube-rbac-proxy-web"
Sep 12 00:35:31.361034 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.360946    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="99f6922b-0b37-4b49-b685-bbfcbb0d8cee" containerName="prometheus"
Sep 12 00:35:31.361034 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.360951    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bba313c-7ee4-46de-b7bf-47e15ebb1add" containerName="console"
Sep 12 00:35:31.361034 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.360955    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="99f6922b-0b37-4b49-b685-bbfcbb0d8cee" containerName="config-reloader"
Sep 12 00:35:31.361034 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.360960    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="99f6922b-0b37-4b49-b685-bbfcbb0d8cee" containerName="thanos-sidecar"
Sep 12 00:35:31.361034 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.360964    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="99f6922b-0b37-4b49-b685-bbfcbb0d8cee" containerName="kube-rbac-proxy"
Sep 12 00:35:31.366504 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.366462    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.369042 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.369021    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 12 00:35:31.369263 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.369244    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 12 00:35:31.369568 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.369535    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 12 00:35:31.369768 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.369708    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-6djp5\""
Sep 12 00:35:31.369858 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.369811    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 12 00:35:31.369858 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.369824    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 12 00:35:31.372379 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.370423    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 12 00:35:31.372379 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.370591    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 12 00:35:31.372379 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.370736    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 12 00:35:31.372379 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.370887    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 12 00:35:31.372379 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.371025    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 12 00:35:31.372662 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.372503    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-etefrcrd49g14\""
Sep 12 00:35:31.372708 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.372667    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 12 00:35:31.373337 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.372922    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 12 00:35:31.375188 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.375163    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 12 00:35:31.377007 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.376992    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 12 00:35:31.379740 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.379717    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 12 00:35:31.511467 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.511437    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b09956e-511a-40ce-94b5-a00ab565695d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.511467 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.511468    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b09956e-511a-40ce-94b5-a00ab565695d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.511722 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.511489    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9b09956e-511a-40ce-94b5-a00ab565695d-web-config\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.511722 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.511507    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9b09956e-511a-40ce-94b5-a00ab565695d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.511722 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.511541    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9b09956e-511a-40ce-94b5-a00ab565695d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.511722 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.511562    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9b09956e-511a-40ce-94b5-a00ab565695d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.511722 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.511585    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b09956e-511a-40ce-94b5-a00ab565695d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.511722 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.511627    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxc5q\" (UniqueName: \"kubernetes.io/projected/9b09956e-511a-40ce-94b5-a00ab565695d-kube-api-access-gxc5q\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.511722 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.511663    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9b09956e-511a-40ce-94b5-a00ab565695d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.511722 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.511688    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9b09956e-511a-40ce-94b5-a00ab565695d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.512120 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.511751    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9b09956e-511a-40ce-94b5-a00ab565695d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.512120 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.511795    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9b09956e-511a-40ce-94b5-a00ab565695d-config\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.512120 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.511823    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b09956e-511a-40ce-94b5-a00ab565695d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.512120 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.511850    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b09956e-511a-40ce-94b5-a00ab565695d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.512120 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.511877    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9b09956e-511a-40ce-94b5-a00ab565695d-config-out\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.512120 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.511901    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b09956e-511a-40ce-94b5-a00ab565695d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.512120 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.511927    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9b09956e-511a-40ce-94b5-a00ab565695d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.512120 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.511959    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9b09956e-511a-40ce-94b5-a00ab565695d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.512120 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.512001    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9b09956e-511a-40ce-94b5-a00ab565695d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.512120 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.512030    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9b09956e-511a-40ce-94b5-a00ab565695d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.612978 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.612884    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b09956e-511a-40ce-94b5-a00ab565695d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.612978 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.612930    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b09956e-511a-40ce-94b5-a00ab565695d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.612978 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.612948    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9b09956e-511a-40ce-94b5-a00ab565695d-web-config\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.612978 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.612966    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9b09956e-511a-40ce-94b5-a00ab565695d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.613283 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.613108    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9b09956e-511a-40ce-94b5-a00ab565695d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.613283 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.613147    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9b09956e-511a-40ce-94b5-a00ab565695d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.613283 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.613182    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b09956e-511a-40ce-94b5-a00ab565695d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.613283 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.613208    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxc5q\" (UniqueName: \"kubernetes.io/projected/9b09956e-511a-40ce-94b5-a00ab565695d-kube-api-access-gxc5q\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.613283 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.613256    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9b09956e-511a-40ce-94b5-a00ab565695d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.613512 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.613288    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9b09956e-511a-40ce-94b5-a00ab565695d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.613512 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.613326    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9b09956e-511a-40ce-94b5-a00ab565695d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.613512 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.613357    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9b09956e-511a-40ce-94b5-a00ab565695d-config\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.613512 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.613381    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b09956e-511a-40ce-94b5-a00ab565695d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.613512 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.613407    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b09956e-511a-40ce-94b5-a00ab565695d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.613512 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.613433    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9b09956e-511a-40ce-94b5-a00ab565695d-config-out\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.613512 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.613458    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b09956e-511a-40ce-94b5-a00ab565695d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.613512 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.613481    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9b09956e-511a-40ce-94b5-a00ab565695d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.613899 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.613519    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9b09956e-511a-40ce-94b5-a00ab565695d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.613899 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.613564    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9b09956e-511a-40ce-94b5-a00ab565695d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.613899 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.613594    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9b09956e-511a-40ce-94b5-a00ab565695d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.613899 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.613633    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9b09956e-511a-40ce-94b5-a00ab565695d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.614112 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.613943    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b09956e-511a-40ce-94b5-a00ab565695d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.614112 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.613960    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b09956e-511a-40ce-94b5-a00ab565695d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.614462 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.614435    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b09956e-511a-40ce-94b5-a00ab565695d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.616423 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.614733    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9b09956e-511a-40ce-94b5-a00ab565695d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.616423 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.615896    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b09956e-511a-40ce-94b5-a00ab565695d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.616590 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.616531    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9b09956e-511a-40ce-94b5-a00ab565695d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.616590 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.616539    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9b09956e-511a-40ce-94b5-a00ab565695d-config-out\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.617043 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.617020    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9b09956e-511a-40ce-94b5-a00ab565695d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.617436 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.617398    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9b09956e-511a-40ce-94b5-a00ab565695d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.618071 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.618018    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9b09956e-511a-40ce-94b5-a00ab565695d-web-config\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.618380 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.618356    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b09956e-511a-40ce-94b5-a00ab565695d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.618516 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.618423    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/9b09956e-511a-40ce-94b5-a00ab565695d-config\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.618563 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.618542    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9b09956e-511a-40ce-94b5-a00ab565695d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.618751 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.618724    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9b09956e-511a-40ce-94b5-a00ab565695d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.618798 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.618752    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b09956e-511a-40ce-94b5-a00ab565695d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.618916 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.618893    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9b09956e-511a-40ce-94b5-a00ab565695d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.618970 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.618956    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9b09956e-511a-40ce-94b5-a00ab565695d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.619503 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.619487    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9b09956e-511a-40ce-94b5-a00ab565695d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.622261 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.622238    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxc5q\" (UniqueName: \"kubernetes.io/projected/9b09956e-511a-40ce-94b5-a00ab565695d-kube-api-access-gxc5q\") pod \"prometheus-k8s-0\" (UID: \"9b09956e-511a-40ce-94b5-a00ab565695d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.678345 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.678308    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:31.742315 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.741619    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99f6922b-0b37-4b49-b685-bbfcbb0d8cee" path="/var/lib/kubelet/pods/99f6922b-0b37-4b49-b685-bbfcbb0d8cee/volumes"
Sep 12 00:35:31.962202 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:31.962176    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 12 00:35:31.964578 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:35:31.964550    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b09956e_511a_40ce_94b5_a00ab565695d.slice/crio-00f4e4f0ddfe8ad96083ce5115c550c4828ed70ccc33d2115e5f1c618c8226d2 WatchSource:0}: Error finding container 00f4e4f0ddfe8ad96083ce5115c550c4828ed70ccc33d2115e5f1c618c8226d2: Status 404 returned error can't find the container with id 00f4e4f0ddfe8ad96083ce5115c550c4828ed70ccc33d2115e5f1c618c8226d2
Sep 12 00:35:32.015438 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:32.015407    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b09956e-511a-40ce-94b5-a00ab565695d","Type":"ContainerStarted","Data":"00f4e4f0ddfe8ad96083ce5115c550c4828ed70ccc33d2115e5f1c618c8226d2"}
Sep 12 00:35:32.922244 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:32.922213    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:35:32.977603 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:32.977521    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:35:33.022239 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:33.022207    2558 generic.go:356] "Generic (PLEG): container finished" podID="9b09956e-511a-40ce-94b5-a00ab565695d" containerID="5f63ff321f06f23fff3d9c860e89d26933979d370bc1c033bf9df67eec84ed0a" exitCode=0
Sep 12 00:35:33.022415 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:33.022294    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b09956e-511a-40ce-94b5-a00ab565695d","Type":"ContainerDied","Data":"5f63ff321f06f23fff3d9c860e89d26933979d370bc1c033bf9df67eec84ed0a"}
Sep 12 00:35:33.073851 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:33.073824    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:35:33.174815 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:33.174793    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:35:33.696034 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:33.696002    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:35:33.796297 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:33.796267    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:35:34.030248 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:34.030164    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b09956e-511a-40ce-94b5-a00ab565695d","Type":"ContainerStarted","Data":"cfc2b3663fe02fd00b310921e0eeec99b21cd14015995dc0ba70ab9cd2f70138"}
Sep 12 00:35:34.030248 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:34.030225    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b09956e-511a-40ce-94b5-a00ab565695d","Type":"ContainerStarted","Data":"c3696e22827546e7ddcb2a9d8988f69cbe9c5fe846de2b6ec9a50c9a46e51a97"}
Sep 12 00:35:34.030248 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:34.030250    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b09956e-511a-40ce-94b5-a00ab565695d","Type":"ContainerStarted","Data":"2e55db731ea27ce494f2f7db12d5bccdb719b90e462ef291badcd2393a3714f1"}
Sep 12 00:35:34.030640 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:34.030273    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b09956e-511a-40ce-94b5-a00ab565695d","Type":"ContainerStarted","Data":"42af3afa3d070bf83b33d33101cc4c40099fc73ed7b96f14f69b66d39a8f8a40"}
Sep 12 00:35:34.030640 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:34.030296    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b09956e-511a-40ce-94b5-a00ab565695d","Type":"ContainerStarted","Data":"11725d10fbc7b387221e555c9ff085c987c1a71d6ce68c82401dc20853427e42"}
Sep 12 00:35:34.030640 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:34.030320    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b09956e-511a-40ce-94b5-a00ab565695d","Type":"ContainerStarted","Data":"266322c4753c1bf030927493f73a9a2412609b7d6446f673256c758c324e04a4"}
Sep 12 00:35:34.055204 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:34.055148    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.034412728 podStartE2EDuration="3.055131892s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.719164ms" isStatefulPod=false podCreationTimestamp="2026-09-12 00:35:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:35:34.054195508 +0000 UTC m=+370.728254374" watchObservedRunningTime="2026-09-12 00:35:34.055131892 +0000 UTC m=+370.729190751"
Sep 12 00:35:36.679338 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:36.679281    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:35:37.667285 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:37.667240    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/746054e5-684e-4c55-824b-6fdde562eb02-metrics-certs\") pod \"network-metrics-daemon-7tkvx\" (UID: \"746054e5-684e-4c55-824b-6fdde562eb02\") " pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:35:37.669709 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:37.669679    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/746054e5-684e-4c55-824b-6fdde562eb02-metrics-certs\") pod \"network-metrics-daemon-7tkvx\" (UID: \"746054e5-684e-4c55-824b-6fdde562eb02\") " pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:35:37.734832 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:37.734809    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8bjlg\""
Sep 12 00:35:37.742794 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:37.742778    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tkvx"
Sep 12 00:35:38.011767 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:38.011740    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7tkvx"]
Sep 12 00:35:38.014277 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:35:38.014246    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod746054e5_684e_4c55_824b_6fdde562eb02.slice/crio-e2a48daf77706bcabd21397566b443b33e13225c52ccb0bf366bd3cd15ecd110 WatchSource:0}: Error finding container e2a48daf77706bcabd21397566b443b33e13225c52ccb0bf366bd3cd15ecd110: Status 404 returned error can't find the container with id e2a48daf77706bcabd21397566b443b33e13225c52ccb0bf366bd3cd15ecd110
Sep 12 00:35:38.016405 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:38.016389    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 12 00:35:38.046212 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:38.046179    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7tkvx" event={"ID":"746054e5-684e-4c55-824b-6fdde562eb02","Type":"ContainerStarted","Data":"e2a48daf77706bcabd21397566b443b33e13225c52ccb0bf366bd3cd15ecd110"}
Sep 12 00:35:40.055463 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:40.055433    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7tkvx" event={"ID":"746054e5-684e-4c55-824b-6fdde562eb02","Type":"ContainerStarted","Data":"422736c846ad1289f0b76b95f9b2e75347916198845fc4e8105b6303337b11ad"}
Sep 12 00:35:40.055850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:40.055480    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7tkvx" event={"ID":"746054e5-684e-4c55-824b-6fdde562eb02","Type":"ContainerStarted","Data":"c1bd2df11fba00813231c96bd232885a17b4ccbfc19868a7b7eeed89583d7c4e"}
Sep 12 00:35:40.071847 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:40.071769    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-7tkvx" podStartSLOduration=375.100496897 podStartE2EDuration="6m16.071750498s" totalImagesPullingTime="971.253601ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:29:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:35:40.070406232 +0000 UTC m=+376.744465092" watchObservedRunningTime="2026-09-12 00:35:40.071750498 +0000 UTC m=+376.745809360"
Sep 12 00:35:40.154465 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:40.154427    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:35:40.205959 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:40.205932    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:35:43.719224 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:43.719181    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:35:43.773725 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:43.773696    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:35:43.962141 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:43.962110    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:35:44.069981 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:44.069874    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:35:44.125530 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:35:44.125494    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:36:05.558728 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:36:05.558690    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:36:25.919132 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:36:25.919098    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:36:25.972299 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:36:25.972270    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:36:31.678668 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:36:31.678627    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:36:31.695150 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:36:31.695124    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:36:32.272616 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:36:32.272589    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 12 00:36:39.458764 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:36:39.458732    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:36:39.509978 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:36:39.509950    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:36:44.284888 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:36:44.284851    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:36:51.382375 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:36:51.382296    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:36:51.433239 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:36:51.433213    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:36:53.729456 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:36:53.729417    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:36:53.779324 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:36:53.779289    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:36:55.214190 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:36:55.214153    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:36:55.314652 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:36:55.314616    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:36:55.414874 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:36:55.414841    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:36:55.466173 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:36:55.466071    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:37:31.550716 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:37:31.550683    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:37:31.644309 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:37:31.644277    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:37:58.264370 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:37:58.264334    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:37:59.037794 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:37:59.037757    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:01.878497 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:01.878464    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:01.978434 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:01.978406    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:03.330801 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:03.330773    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:03.431725 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:03.431695    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:03.827673 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:03.827635    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:03.946983 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:03.946950    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:05.544408 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:05.544376    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:05.723808 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:05.723782    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:08.666490 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:08.666456    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:08.766476 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:08.766443    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:09.376795 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:09.376765    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:12.933640 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:12.933594    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:13.034860 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:13.034830    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:13.738239 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:13.738207    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:20.076610 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:20.076520    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:20.177626 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:20.177588    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:22.097699 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:22.097657    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jgslq"]
Sep 12 00:38:22.102759 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:22.102722    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jgslq"
Sep 12 00:38:22.105305 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:22.105285    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 12 00:38:22.105305 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:22.105299    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 12 00:38:22.106248 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:22.106228    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-rbk55\""
Sep 12 00:38:22.109013 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:22.108992    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jgslq"]
Sep 12 00:38:22.149036 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:22.149001    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnmtb\" (UniqueName: \"kubernetes.io/projected/6686840f-1e9a-4fdf-87d0-0c7222538f54-kube-api-access-hnmtb\") pod \"openshift-lws-operator-fd8ccff4c-jgslq\" (UID: \"6686840f-1e9a-4fdf-87d0-0c7222538f54\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jgslq"
Sep 12 00:38:22.149185 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:22.149067    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6686840f-1e9a-4fdf-87d0-0c7222538f54-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jgslq\" (UID: \"6686840f-1e9a-4fdf-87d0-0c7222538f54\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jgslq"
Sep 12 00:38:22.249433 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:22.249389    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnmtb\" (UniqueName: \"kubernetes.io/projected/6686840f-1e9a-4fdf-87d0-0c7222538f54-kube-api-access-hnmtb\") pod \"openshift-lws-operator-fd8ccff4c-jgslq\" (UID: \"6686840f-1e9a-4fdf-87d0-0c7222538f54\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jgslq"
Sep 12 00:38:22.249615 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:22.249464    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6686840f-1e9a-4fdf-87d0-0c7222538f54-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jgslq\" (UID: \"6686840f-1e9a-4fdf-87d0-0c7222538f54\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jgslq"
Sep 12 00:38:22.249765 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:22.249750    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6686840f-1e9a-4fdf-87d0-0c7222538f54-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jgslq\" (UID: \"6686840f-1e9a-4fdf-87d0-0c7222538f54\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jgslq"
Sep 12 00:38:22.257731 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:22.257694    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnmtb\" (UniqueName: \"kubernetes.io/projected/6686840f-1e9a-4fdf-87d0-0c7222538f54-kube-api-access-hnmtb\") pod \"openshift-lws-operator-fd8ccff4c-jgslq\" (UID: \"6686840f-1e9a-4fdf-87d0-0c7222538f54\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jgslq"
Sep 12 00:38:22.433318 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:22.433232    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jgslq"
Sep 12 00:38:22.705354 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:22.705332    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jgslq"]
Sep 12 00:38:22.707497 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:38:22.707471    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6686840f_1e9a_4fdf_87d0_0c7222538f54.slice/crio-cd0bc58341be77e870163d888d3e871d73c5762c9cb19244f4a06db0b64fd6b1 WatchSource:0}: Error finding container cd0bc58341be77e870163d888d3e871d73c5762c9cb19244f4a06db0b64fd6b1: Status 404 returned error can't find the container with id cd0bc58341be77e870163d888d3e871d73c5762c9cb19244f4a06db0b64fd6b1
Sep 12 00:38:23.688937 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:23.688865    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jgslq" event={"ID":"6686840f-1e9a-4fdf-87d0-0c7222538f54","Type":"ContainerStarted","Data":"cd0bc58341be77e870163d888d3e871d73c5762c9cb19244f4a06db0b64fd6b1"}
Sep 12 00:38:25.700407 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:25.700370    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jgslq" event={"ID":"6686840f-1e9a-4fdf-87d0-0c7222538f54","Type":"ContainerStarted","Data":"9b251420acd9f5d9671d49f04bdc8d0496f15944eb5ae760c86942d892be0341"}
Sep 12 00:38:25.716561 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:25.716514    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jgslq" podStartSLOduration=1.231827146 podStartE2EDuration="3.716498896s" totalImagesPullingTime="2.48467175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:38:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:38:25.716171282 +0000 UTC m=+542.390230143" watchObservedRunningTime="2026-09-12 00:38:25.716498896 +0000 UTC m=+542.390557748"
Sep 12 00:38:26.369327 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:26.369291    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:26.468787 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:26.468754    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:28.211174 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:28.211144    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:31.365509 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:31.365476    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:31.415932 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:31.415904    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:36.467547 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:36.467508    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:36.508696 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:36.508660    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7bc7647c9b-82h5h"]
Sep 12 00:38:36.512069 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:36.512047    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7bc7647c9b-82h5h"
Sep 12 00:38:36.515942 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:36.515922    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 12 00:38:36.516225 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:36.516210    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 12 00:38:36.516291 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:36.516213    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-rkprs\""
Sep 12 00:38:36.516291 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:36.516267    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 12 00:38:36.521640 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:36.521620    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7bc7647c9b-82h5h"]
Sep 12 00:38:36.564255 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:36.564230    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:36.573436 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:36.573408    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/030ba73a-88ac-488a-ac3e-7b12fd28bf27-metrics-cert\") pod \"lws-controller-manager-7bc7647c9b-82h5h\" (UID: \"030ba73a-88ac-488a-ac3e-7b12fd28bf27\") " pod="openshift-lws-operator/lws-controller-manager-7bc7647c9b-82h5h"
Sep 12 00:38:36.573516 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:36.573440    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/030ba73a-88ac-488a-ac3e-7b12fd28bf27-cert\") pod \"lws-controller-manager-7bc7647c9b-82h5h\" (UID: \"030ba73a-88ac-488a-ac3e-7b12fd28bf27\") " pod="openshift-lws-operator/lws-controller-manager-7bc7647c9b-82h5h"
Sep 12 00:38:36.573516 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:36.573479    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq7vv\" (UniqueName: \"kubernetes.io/projected/030ba73a-88ac-488a-ac3e-7b12fd28bf27-kube-api-access-sq7vv\") pod \"lws-controller-manager-7bc7647c9b-82h5h\" (UID: \"030ba73a-88ac-488a-ac3e-7b12fd28bf27\") " pod="openshift-lws-operator/lws-controller-manager-7bc7647c9b-82h5h"
Sep 12 00:38:36.573611 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:36.573592    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/030ba73a-88ac-488a-ac3e-7b12fd28bf27-manager-config\") pod \"lws-controller-manager-7bc7647c9b-82h5h\" (UID: \"030ba73a-88ac-488a-ac3e-7b12fd28bf27\") " pod="openshift-lws-operator/lws-controller-manager-7bc7647c9b-82h5h"
Sep 12 00:38:36.674779 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:36.674747    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq7vv\" (UniqueName: \"kubernetes.io/projected/030ba73a-88ac-488a-ac3e-7b12fd28bf27-kube-api-access-sq7vv\") pod \"lws-controller-manager-7bc7647c9b-82h5h\" (UID: \"030ba73a-88ac-488a-ac3e-7b12fd28bf27\") " pod="openshift-lws-operator/lws-controller-manager-7bc7647c9b-82h5h"
Sep 12 00:38:36.674930 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:36.674797    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/030ba73a-88ac-488a-ac3e-7b12fd28bf27-manager-config\") pod \"lws-controller-manager-7bc7647c9b-82h5h\" (UID: \"030ba73a-88ac-488a-ac3e-7b12fd28bf27\") " pod="openshift-lws-operator/lws-controller-manager-7bc7647c9b-82h5h"
Sep 12 00:38:36.674930 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:36.674828    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/030ba73a-88ac-488a-ac3e-7b12fd28bf27-metrics-cert\") pod \"lws-controller-manager-7bc7647c9b-82h5h\" (UID: \"030ba73a-88ac-488a-ac3e-7b12fd28bf27\") " pod="openshift-lws-operator/lws-controller-manager-7bc7647c9b-82h5h"
Sep 12 00:38:36.674930 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:36.674843    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/030ba73a-88ac-488a-ac3e-7b12fd28bf27-cert\") pod \"lws-controller-manager-7bc7647c9b-82h5h\" (UID: \"030ba73a-88ac-488a-ac3e-7b12fd28bf27\") " pod="openshift-lws-operator/lws-controller-manager-7bc7647c9b-82h5h"
Sep 12 00:38:36.675586 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:36.675566    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/030ba73a-88ac-488a-ac3e-7b12fd28bf27-manager-config\") pod \"lws-controller-manager-7bc7647c9b-82h5h\" (UID: \"030ba73a-88ac-488a-ac3e-7b12fd28bf27\") " pod="openshift-lws-operator/lws-controller-manager-7bc7647c9b-82h5h"
Sep 12 00:38:36.677351 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:36.677334    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/030ba73a-88ac-488a-ac3e-7b12fd28bf27-metrics-cert\") pod \"lws-controller-manager-7bc7647c9b-82h5h\" (UID: \"030ba73a-88ac-488a-ac3e-7b12fd28bf27\") " pod="openshift-lws-operator/lws-controller-manager-7bc7647c9b-82h5h"
Sep 12 00:38:36.677415 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:36.677353    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/030ba73a-88ac-488a-ac3e-7b12fd28bf27-cert\") pod \"lws-controller-manager-7bc7647c9b-82h5h\" (UID: \"030ba73a-88ac-488a-ac3e-7b12fd28bf27\") " pod="openshift-lws-operator/lws-controller-manager-7bc7647c9b-82h5h"
Sep 12 00:38:36.685208 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:36.685184    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq7vv\" (UniqueName: \"kubernetes.io/projected/030ba73a-88ac-488a-ac3e-7b12fd28bf27-kube-api-access-sq7vv\") pod \"lws-controller-manager-7bc7647c9b-82h5h\" (UID: \"030ba73a-88ac-488a-ac3e-7b12fd28bf27\") " pod="openshift-lws-operator/lws-controller-manager-7bc7647c9b-82h5h"
Sep 12 00:38:36.822163 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:36.822048    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7bc7647c9b-82h5h"
Sep 12 00:38:37.108276 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:37.108188    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7bc7647c9b-82h5h"]
Sep 12 00:38:37.111833 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:38:37.111807    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod030ba73a_88ac_488a_ac3e_7b12fd28bf27.slice/crio-3e49da063b235233d99fe4d2832dba862eaed75c09dfb44ddaf79a9cfdb41c7d WatchSource:0}: Error finding container 3e49da063b235233d99fe4d2832dba862eaed75c09dfb44ddaf79a9cfdb41c7d: Status 404 returned error can't find the container with id 3e49da063b235233d99fe4d2832dba862eaed75c09dfb44ddaf79a9cfdb41c7d
Sep 12 00:38:37.747251 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:37.747217    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7bc7647c9b-82h5h" event={"ID":"030ba73a-88ac-488a-ac3e-7b12fd28bf27","Type":"ContainerStarted","Data":"3e49da063b235233d99fe4d2832dba862eaed75c09dfb44ddaf79a9cfdb41c7d"}
Sep 12 00:38:40.870452 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:40.870416    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:40.976355 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:40.976327    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:41.479315 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:41.479280    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:41.714322 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:41.714275    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:41.817558 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:41.817466    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:41.900633 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:41.900600    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67b7744c54-rgj84"]
Sep 12 00:38:41.907126 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:41.907106    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-67b7744c54-rgj84"
Sep 12 00:38:41.913257 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:41.913236    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 12 00:38:41.913505 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:41.913490    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 12 00:38:41.913570 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:41.913515    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 12 00:38:41.913722 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:41.913707    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-g56gz\""
Sep 12 00:38:41.914298 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:41.914281    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 12 00:38:41.926252 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:41.926218    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67b7744c54-rgj84"]
Sep 12 00:38:42.021522 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:42.021488    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9bc3e216-bd52-4f60-9ffa-b851f019174f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67b7744c54-rgj84\" (UID: \"9bc3e216-bd52-4f60-9ffa-b851f019174f\") " pod="opendatahub/opendatahub-operator-controller-manager-67b7744c54-rgj84"
Sep 12 00:38:42.021693 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:42.021531    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9bc3e216-bd52-4f60-9ffa-b851f019174f-webhook-cert\") pod \"opendatahub-operator-controller-manager-67b7744c54-rgj84\" (UID: \"9bc3e216-bd52-4f60-9ffa-b851f019174f\") " pod="opendatahub/opendatahub-operator-controller-manager-67b7744c54-rgj84"
Sep 12 00:38:42.021693 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:42.021580    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9bc3e216-bd52-4f60-9ffa-b851f019174f-tmp\") pod \"opendatahub-operator-controller-manager-67b7744c54-rgj84\" (UID: \"9bc3e216-bd52-4f60-9ffa-b851f019174f\") " pod="opendatahub/opendatahub-operator-controller-manager-67b7744c54-rgj84"
Sep 12 00:38:42.021693 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:42.021605    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njzxv\" (UniqueName: \"kubernetes.io/projected/9bc3e216-bd52-4f60-9ffa-b851f019174f-kube-api-access-njzxv\") pod \"opendatahub-operator-controller-manager-67b7744c54-rgj84\" (UID: \"9bc3e216-bd52-4f60-9ffa-b851f019174f\") " pod="opendatahub/opendatahub-operator-controller-manager-67b7744c54-rgj84"
Sep 12 00:38:42.021693 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:42.021626    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9bc3e216-bd52-4f60-9ffa-b851f019174f-manifests\") pod \"opendatahub-operator-controller-manager-67b7744c54-rgj84\" (UID: \"9bc3e216-bd52-4f60-9ffa-b851f019174f\") " pod="opendatahub/opendatahub-operator-controller-manager-67b7744c54-rgj84"
Sep 12 00:38:42.123178 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:42.123068    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9bc3e216-bd52-4f60-9ffa-b851f019174f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67b7744c54-rgj84\" (UID: \"9bc3e216-bd52-4f60-9ffa-b851f019174f\") " pod="opendatahub/opendatahub-operator-controller-manager-67b7744c54-rgj84"
Sep 12 00:38:42.123178 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:42.123143    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9bc3e216-bd52-4f60-9ffa-b851f019174f-webhook-cert\") pod \"opendatahub-operator-controller-manager-67b7744c54-rgj84\" (UID: \"9bc3e216-bd52-4f60-9ffa-b851f019174f\") " pod="opendatahub/opendatahub-operator-controller-manager-67b7744c54-rgj84"
Sep 12 00:38:42.123178 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:42.123171    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9bc3e216-bd52-4f60-9ffa-b851f019174f-tmp\") pod \"opendatahub-operator-controller-manager-67b7744c54-rgj84\" (UID: \"9bc3e216-bd52-4f60-9ffa-b851f019174f\") " pod="opendatahub/opendatahub-operator-controller-manager-67b7744c54-rgj84"
Sep 12 00:38:42.123401 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:42.123197    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njzxv\" (UniqueName: \"kubernetes.io/projected/9bc3e216-bd52-4f60-9ffa-b851f019174f-kube-api-access-njzxv\") pod \"opendatahub-operator-controller-manager-67b7744c54-rgj84\" (UID: \"9bc3e216-bd52-4f60-9ffa-b851f019174f\") " pod="opendatahub/opendatahub-operator-controller-manager-67b7744c54-rgj84"
Sep 12 00:38:42.123401 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:42.123225    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9bc3e216-bd52-4f60-9ffa-b851f019174f-manifests\") pod \"opendatahub-operator-controller-manager-67b7744c54-rgj84\" (UID: \"9bc3e216-bd52-4f60-9ffa-b851f019174f\") " pod="opendatahub/opendatahub-operator-controller-manager-67b7744c54-rgj84"
Sep 12 00:38:42.123700 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:42.123671    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9bc3e216-bd52-4f60-9ffa-b851f019174f-manifests\") pod \"opendatahub-operator-controller-manager-67b7744c54-rgj84\" (UID: \"9bc3e216-bd52-4f60-9ffa-b851f019174f\") " pod="opendatahub/opendatahub-operator-controller-manager-67b7744c54-rgj84"
Sep 12 00:38:42.125591 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:42.125565    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9bc3e216-bd52-4f60-9ffa-b851f019174f-tmp\") pod \"opendatahub-operator-controller-manager-67b7744c54-rgj84\" (UID: \"9bc3e216-bd52-4f60-9ffa-b851f019174f\") " pod="opendatahub/opendatahub-operator-controller-manager-67b7744c54-rgj84"
Sep 12 00:38:42.125825 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:42.125802    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9bc3e216-bd52-4f60-9ffa-b851f019174f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67b7744c54-rgj84\" (UID: \"9bc3e216-bd52-4f60-9ffa-b851f019174f\") " pod="opendatahub/opendatahub-operator-controller-manager-67b7744c54-rgj84"
Sep 12 00:38:42.125962 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:42.125941    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9bc3e216-bd52-4f60-9ffa-b851f019174f-webhook-cert\") pod \"opendatahub-operator-controller-manager-67b7744c54-rgj84\" (UID: \"9bc3e216-bd52-4f60-9ffa-b851f019174f\") " pod="opendatahub/opendatahub-operator-controller-manager-67b7744c54-rgj84"
Sep 12 00:38:42.135907 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:42.135886    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njzxv\" (UniqueName: \"kubernetes.io/projected/9bc3e216-bd52-4f60-9ffa-b851f019174f-kube-api-access-njzxv\") pod \"opendatahub-operator-controller-manager-67b7744c54-rgj84\" (UID: \"9bc3e216-bd52-4f60-9ffa-b851f019174f\") " pod="opendatahub/opendatahub-operator-controller-manager-67b7744c54-rgj84"
Sep 12 00:38:42.196971 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:42.196926    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:42.217846 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:42.217827    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-67b7744c54-rgj84"
Sep 12 00:38:42.299354 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:42.299323    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:42.397461 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:42.397383    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:42.497285 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:42.497248    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:42.497817 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:42.497797    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67b7744c54-rgj84"]
Sep 12 00:38:42.500371 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:38:42.500346    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bc3e216_bd52_4f60_9ffa_b851f019174f.slice/crio-fb56a089e69f44dd447372b7f9b49a4e9ff4f9322b027272389e3ec10a92bc68 WatchSource:0}: Error finding container fb56a089e69f44dd447372b7f9b49a4e9ff4f9322b027272389e3ec10a92bc68: Status 404 returned error can't find the container with id fb56a089e69f44dd447372b7f9b49a4e9ff4f9322b027272389e3ec10a92bc68
Sep 12 00:38:42.561224 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:42.561174    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:42.665140 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:42.665038    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:42.767527 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:42.767499    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67b7744c54-rgj84" event={"ID":"9bc3e216-bd52-4f60-9ffa-b851f019174f","Type":"ContainerStarted","Data":"fb56a089e69f44dd447372b7f9b49a4e9ff4f9322b027272389e3ec10a92bc68"}
Sep 12 00:38:45.782564 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:45.782484    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67b7744c54-rgj84" event={"ID":"9bc3e216-bd52-4f60-9ffa-b851f019174f","Type":"ContainerStarted","Data":"9ea0afb1325f16c4ef354d9f193d71f52a69b1d1eb3997596a884ecb7f18d735"}
Sep 12 00:38:46.788695 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:46.788660    2558 generic.go:356] "Generic (PLEG): container finished" podID="9bc3e216-bd52-4f60-9ffa-b851f019174f" containerID="9ea0afb1325f16c4ef354d9f193d71f52a69b1d1eb3997596a884ecb7f18d735" exitCode=0
Sep 12 00:38:46.789101 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:46.788744    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67b7744c54-rgj84" event={"ID":"9bc3e216-bd52-4f60-9ffa-b851f019174f","Type":"ContainerDied","Data":"9ea0afb1325f16c4ef354d9f193d71f52a69b1d1eb3997596a884ecb7f18d735"}
Sep 12 00:38:47.795307 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:47.795271    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67b7744c54-rgj84" event={"ID":"9bc3e216-bd52-4f60-9ffa-b851f019174f","Type":"ContainerStarted","Data":"70c7af9c4204806080daa1141ea351fae812f3fb50de2e70562c8642c2aa2f6d"}
Sep 12 00:38:47.795713 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:47.795350    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-67b7744c54-rgj84"
Sep 12 00:38:47.820433 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:47.820383    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-67b7744c54-rgj84" podStartSLOduration=2.416789575 podStartE2EDuration="6.820367164s" totalImagesPullingTime="3.331461852s" totalInitContainerRuntime="1.072115737s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:38:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:38:47.819500679 +0000 UTC m=+564.493559539" watchObservedRunningTime="2026-09-12 00:38:47.820367164 +0000 UTC m=+564.494426023"
Sep 12 00:38:52.465255 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:52.465222    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:52.566100 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:52.566055    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:38:58.802206 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:58.802174    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-67b7744c54-rgj84"
Sep 12 00:38:58.845994 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:58.845956    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7bc7647c9b-82h5h" event={"ID":"030ba73a-88ac-488a-ac3e-7b12fd28bf27","Type":"ContainerStarted","Data":"3fb9c141702b25d36481d1139f31ff50df300c12c64bab7f0ac72e39402af034"}
Sep 12 00:38:58.846224 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:58.846018    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7bc7647c9b-82h5h"
Sep 12 00:38:58.863630 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:38:58.863569    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7bc7647c9b-82h5h" podStartSLOduration=2.190415012 podStartE2EDuration="22.863552977s" totalImagesPullingTime="20.673137965s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:38:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:38:58.861653913 +0000 UTC m=+575.535712773" watchObservedRunningTime="2026-09-12 00:38:58.863552977 +0000 UTC m=+575.537611873"
Sep 12 00:39:00.876000 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:00.875962    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:39:00.987892 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:00.987859    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:39:01.378730 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:01.378694    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:39:01.406394 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:01.406364    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-797549bd89-h8pbs"]
Sep 12 00:39:01.410395 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:01.410370    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-797549bd89-h8pbs"
Sep 12 00:39:01.415289 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:01.415269    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 12 00:39:01.415404 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:01.415300    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 12 00:39:01.422235 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:01.422219    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-btljl\""
Sep 12 00:39:01.489765 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:01.489735    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef6df39b-1ed0-4873-a049-2bc77833c2a3-tmp\") pod \"kube-auth-proxy-797549bd89-h8pbs\" (UID: \"ef6df39b-1ed0-4873-a049-2bc77833c2a3\") " pod="openshift-ingress/kube-auth-proxy-797549bd89-h8pbs"
Sep 12 00:39:01.489946 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:01.489795    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxmhx\" (UniqueName: \"kubernetes.io/projected/ef6df39b-1ed0-4873-a049-2bc77833c2a3-kube-api-access-lxmhx\") pod \"kube-auth-proxy-797549bd89-h8pbs\" (UID: \"ef6df39b-1ed0-4873-a049-2bc77833c2a3\") " pod="openshift-ingress/kube-auth-proxy-797549bd89-h8pbs"
Sep 12 00:39:01.489946 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:01.489833    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef6df39b-1ed0-4873-a049-2bc77833c2a3-tls-certs\") pod \"kube-auth-proxy-797549bd89-h8pbs\" (UID: \"ef6df39b-1ed0-4873-a049-2bc77833c2a3\") " pod="openshift-ingress/kube-auth-proxy-797549bd89-h8pbs"
Sep 12 00:39:01.513663 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:01.513632    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-797549bd89-h8pbs"]
Sep 12 00:39:01.591209 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:01.591171    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef6df39b-1ed0-4873-a049-2bc77833c2a3-tmp\") pod \"kube-auth-proxy-797549bd89-h8pbs\" (UID: \"ef6df39b-1ed0-4873-a049-2bc77833c2a3\") " pod="openshift-ingress/kube-auth-proxy-797549bd89-h8pbs"
Sep 12 00:39:01.591408 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:01.591222    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxmhx\" (UniqueName: \"kubernetes.io/projected/ef6df39b-1ed0-4873-a049-2bc77833c2a3-kube-api-access-lxmhx\") pod \"kube-auth-proxy-797549bd89-h8pbs\" (UID: \"ef6df39b-1ed0-4873-a049-2bc77833c2a3\") " pod="openshift-ingress/kube-auth-proxy-797549bd89-h8pbs"
Sep 12 00:39:01.591408 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:01.591251    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef6df39b-1ed0-4873-a049-2bc77833c2a3-tls-certs\") pod \"kube-auth-proxy-797549bd89-h8pbs\" (UID: \"ef6df39b-1ed0-4873-a049-2bc77833c2a3\") " pod="openshift-ingress/kube-auth-proxy-797549bd89-h8pbs"
Sep 12 00:39:01.593813 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:01.593792    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef6df39b-1ed0-4873-a049-2bc77833c2a3-tmp\") pod \"kube-auth-proxy-797549bd89-h8pbs\" (UID: \"ef6df39b-1ed0-4873-a049-2bc77833c2a3\") " pod="openshift-ingress/kube-auth-proxy-797549bd89-h8pbs"
Sep 12 00:39:01.593961 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:01.593942    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:39:01.594019 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:01.593981    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef6df39b-1ed0-4873-a049-2bc77833c2a3-tls-certs\") pod \"kube-auth-proxy-797549bd89-h8pbs\" (UID: \"ef6df39b-1ed0-4873-a049-2bc77833c2a3\") " pod="openshift-ingress/kube-auth-proxy-797549bd89-h8pbs"
Sep 12 00:39:01.604411 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:01.604390    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxmhx\" (UniqueName: \"kubernetes.io/projected/ef6df39b-1ed0-4873-a049-2bc77833c2a3-kube-api-access-lxmhx\") pod \"kube-auth-proxy-797549bd89-h8pbs\" (UID: \"ef6df39b-1ed0-4873-a049-2bc77833c2a3\") " pod="openshift-ingress/kube-auth-proxy-797549bd89-h8pbs"
Sep 12 00:39:01.721342 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:01.721310    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-797549bd89-h8pbs"
Sep 12 00:39:01.995691 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:01.995562    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-797549bd89-h8pbs"]
Sep 12 00:39:01.998252 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:39:01.998227    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef6df39b_1ed0_4873_a049_2bc77833c2a3.slice/crio-ff0d527544a9aa004a6f411dfac04c8b16559ffe98d3045e4f300addbe9f974b WatchSource:0}: Error finding container ff0d527544a9aa004a6f411dfac04c8b16559ffe98d3045e4f300addbe9f974b: Status 404 returned error can't find the container with id ff0d527544a9aa004a6f411dfac04c8b16559ffe98d3045e4f300addbe9f974b
Sep 12 00:39:02.864006 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:02.863966    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-797549bd89-h8pbs" event={"ID":"ef6df39b-1ed0-4873-a049-2bc77833c2a3","Type":"ContainerStarted","Data":"ff0d527544a9aa004a6f411dfac04c8b16559ffe98d3045e4f300addbe9f974b"}
Sep 12 00:39:03.026335 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:03.026300    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:39:03.089680 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:03.089630    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:39:04.020574 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:04.020530    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:39:05.880177 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:05.880140    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-797549bd89-h8pbs" event={"ID":"ef6df39b-1ed0-4873-a049-2bc77833c2a3","Type":"ContainerStarted","Data":"82f41c1b2a2e1a6d68fa71801b96714153f84d10197cd95f395071aeab57d607"}
Sep 12 00:39:05.895639 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:05.895593    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-797549bd89-h8pbs" podStartSLOduration=1.870866659 podStartE2EDuration="4.895567793s" totalImagesPullingTime="3.024701134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:39:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:39:05.895062303 +0000 UTC m=+582.569121185" watchObservedRunningTime="2026-09-12 00:39:05.895567793 +0000 UTC m=+582.569626651"
Sep 12 00:39:07.229166 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:07.229134    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:39:07.302095 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:07.302062    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:39:07.943044 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:07.943008    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:39:08.059025 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:08.058997    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:39:08.833170 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:08.833135    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:39:09.042251 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:09.042215    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:39:09.435667 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:09.435634    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:39:09.852877 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:09.852801    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7bc7647c9b-82h5h"
Sep 12 00:39:12.086418 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:12.086382    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:39:13.216780 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:13.216746    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:39:18.126318 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:18.126280    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:39:18.285816 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:18.285784    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:39:21.902627 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:21.902601    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:39:22.031389 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:22.031358    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:39:22.407233 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:22.407205    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:39:22.489791 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:22.489763    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:39:23.216041 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:23.216007    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:39:23.322208 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:23.322170    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:39:23.479046 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:23.478972    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:39:23.720809 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:23.720712    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-czf8h_dd90996a-5428-4ccf-a154-c0aa8def9789/console-operator/1.log"
Sep 12 00:39:23.728487 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:39:23.722219    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-czf8h_dd90996a-5428-4ccf-a154-c0aa8def9789/console-operator/1.log"
Sep 12 00:40:02.876405 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:02.876371    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:02.976564 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:02.976538    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:12.946351 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:12.946314    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:13.055338 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:13.055282    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:26.523575 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:26.523537    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:29.111535 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:29.111496    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:29.516464 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:29.516421    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:29.898380 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:29.898287    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:30.750680 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:30.750640    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:32.549013 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:32.548978    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:33.500005 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:33.499973    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:35.105370 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:35.105336    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:35.344607 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:35.344574    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:35.552731 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:35.552696    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:37.550362 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:37.550334    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:37.646174 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:37.646137    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:38.262182 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:38.262140    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:39.167165 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:39.167128    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:39.259957 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:39.259844    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:39.912984 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:39.912944    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:39.967440 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:39.967403    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:43.628243 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:43.628207    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:43.732592 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:43.732555    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:43.914064 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:43.913971    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:44.017470 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:44.017425    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:44.118184 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:44.118153    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:44.218895 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:44.218865    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:45.101855 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:45.101822    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:45.202231 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:45.202198    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:50.757308 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:50.757269    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:50.886038 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:50.886003    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:55.550837 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:55.550803    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:55.650418 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:55.650382    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:55.923541 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:55.923463    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:56.018293 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:56.018261    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:58.144724 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:58.144696    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:40:58.245628 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:40:58.245598    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:41:03.565511 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:03.565473    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:41:03.666396 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:03.666341    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:41:03.952183 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:03.952154    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:41:04.045902 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:04.045876    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:41:04.340809 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:04.340732    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:41:04.442255 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:04.442225    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:41:10.551838 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:10.551783    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:41:10.653483 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:10.653421    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:41:15.368310 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:15.368234    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:41:15.469971 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:15.469937    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:41:18.878868 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:18.878831    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:41:19.341032 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:19.340996    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:41:35.487602 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:35.487564    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hfg9d"]
Sep 12 00:41:35.490875 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:35.490849    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:41:35.491016 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:35.491003    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-hfg9d"
Sep 12 00:41:35.493293 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:35.493270    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 12 00:41:35.494238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:35.494220    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mhmr6\""
Sep 12 00:41:35.494356 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:35.494242    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 12 00:41:35.494356 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:35.494255    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 12 00:41:35.500105 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:35.500065    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hfg9d"]
Sep 12 00:41:35.541817 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:35.541776    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wnjh\" (UniqueName: \"kubernetes.io/projected/8a5d3ce5-76c2-4f07-935f-1bba014a345a-kube-api-access-2wnjh\") pod \"limitador-limitador-8588749977-hfg9d\" (UID: \"8a5d3ce5-76c2-4f07-935f-1bba014a345a\") " pod="kuadrant-system/limitador-limitador-8588749977-hfg9d"
Sep 12 00:41:35.542000 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:35.541827    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8a5d3ce5-76c2-4f07-935f-1bba014a345a-config-file\") pod \"limitador-limitador-8588749977-hfg9d\" (UID: \"8a5d3ce5-76c2-4f07-935f-1bba014a345a\") " pod="kuadrant-system/limitador-limitador-8588749977-hfg9d"
Sep 12 00:41:35.591179 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:35.591141    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:41:35.591376 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:35.591261    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hfg9d"]
Sep 12 00:41:35.642643 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:35.642609    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wnjh\" (UniqueName: \"kubernetes.io/projected/8a5d3ce5-76c2-4f07-935f-1bba014a345a-kube-api-access-2wnjh\") pod \"limitador-limitador-8588749977-hfg9d\" (UID: \"8a5d3ce5-76c2-4f07-935f-1bba014a345a\") " pod="kuadrant-system/limitador-limitador-8588749977-hfg9d"
Sep 12 00:41:35.642817 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:35.642655    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8a5d3ce5-76c2-4f07-935f-1bba014a345a-config-file\") pod \"limitador-limitador-8588749977-hfg9d\" (UID: \"8a5d3ce5-76c2-4f07-935f-1bba014a345a\") " pod="kuadrant-system/limitador-limitador-8588749977-hfg9d"
Sep 12 00:41:35.643288 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:35.643269    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8a5d3ce5-76c2-4f07-935f-1bba014a345a-config-file\") pod \"limitador-limitador-8588749977-hfg9d\" (UID: \"8a5d3ce5-76c2-4f07-935f-1bba014a345a\") " pod="kuadrant-system/limitador-limitador-8588749977-hfg9d"
Sep 12 00:41:35.650422 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:35.650400    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wnjh\" (UniqueName: \"kubernetes.io/projected/8a5d3ce5-76c2-4f07-935f-1bba014a345a-kube-api-access-2wnjh\") pod \"limitador-limitador-8588749977-hfg9d\" (UID: \"8a5d3ce5-76c2-4f07-935f-1bba014a345a\") " pod="kuadrant-system/limitador-limitador-8588749977-hfg9d"
Sep 12 00:41:35.692096 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:35.692061    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:41:35.801797 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:35.801712    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-hfg9d"
Sep 12 00:41:36.072176 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:36.072156    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hfg9d"]
Sep 12 00:41:36.074485 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:41:36.074453    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a5d3ce5_76c2_4f07_935f_1bba014a345a.slice/crio-f895399862d86f46b6e8266dc981b447e24abe3be67a0c91a169f037bbe9b37c WatchSource:0}: Error finding container f895399862d86f46b6e8266dc981b447e24abe3be67a0c91a169f037bbe9b37c: Status 404 returned error can't find the container with id f895399862d86f46b6e8266dc981b447e24abe3be67a0c91a169f037bbe9b37c
Sep 12 00:41:36.076527 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:36.076506    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 12 00:41:36.319874 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:36.319837    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:41:36.396704 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:36.396627    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:41:36.527974 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:36.527940    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-hfg9d" event={"ID":"8a5d3ce5-76c2-4f07-935f-1bba014a345a","Type":"ContainerStarted","Data":"f895399862d86f46b6e8266dc981b447e24abe3be67a0c91a169f037bbe9b37c"}
Sep 12 00:41:38.538235 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:38.538203    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-hfg9d" event={"ID":"8a5d3ce5-76c2-4f07-935f-1bba014a345a","Type":"ContainerStarted","Data":"4747f8c8d4f674dad6d8e7a589e22f0aa8f4eaa0a2a20552254f7ad615fbbe9f"}
Sep 12 00:41:38.538623 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:38.538313    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-hfg9d"
Sep 12 00:41:38.553366 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:38.553326    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-hfg9d" podStartSLOduration=1.256593753 podStartE2EDuration="3.553313169s" totalImagesPullingTime="2.296719416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:41:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:41:38.552159282 +0000 UTC m=+735.226218140" watchObservedRunningTime="2026-09-12 00:41:38.553313169 +0000 UTC m=+735.227372027"
Sep 12 00:41:48.734328 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:48.734293    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hfg9d"]
Sep 12 00:41:48.735137 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:48.734507    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-hfg9d" podUID="8a5d3ce5-76c2-4f07-935f-1bba014a345a" containerName="limitador" containerID="cri-o://4747f8c8d4f674dad6d8e7a589e22f0aa8f4eaa0a2a20552254f7ad615fbbe9f" gracePeriod=30
Sep 12 00:41:48.735436 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:48.735160    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-hfg9d"
Sep 12 00:41:49.274022 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:49.274000    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-hfg9d"
Sep 12 00:41:49.367856 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:49.367765    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8a5d3ce5-76c2-4f07-935f-1bba014a345a-config-file\") pod \"8a5d3ce5-76c2-4f07-935f-1bba014a345a\" (UID: \"8a5d3ce5-76c2-4f07-935f-1bba014a345a\") "
Sep 12 00:41:49.367856 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:49.367846    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2wnjh\" (UniqueName: \"kubernetes.io/projected/8a5d3ce5-76c2-4f07-935f-1bba014a345a-kube-api-access-2wnjh\") pod \"8a5d3ce5-76c2-4f07-935f-1bba014a345a\" (UID: \"8a5d3ce5-76c2-4f07-935f-1bba014a345a\") "
Sep 12 00:41:49.368227 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:49.368202    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a5d3ce5-76c2-4f07-935f-1bba014a345a-config-file" (OuterVolumeSpecName: "config-file") pod "8a5d3ce5-76c2-4f07-935f-1bba014a345a" (UID: "8a5d3ce5-76c2-4f07-935f-1bba014a345a"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:41:49.370146 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:49.370116    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a5d3ce5-76c2-4f07-935f-1bba014a345a-kube-api-access-2wnjh" (OuterVolumeSpecName: "kube-api-access-2wnjh") pod "8a5d3ce5-76c2-4f07-935f-1bba014a345a" (UID: "8a5d3ce5-76c2-4f07-935f-1bba014a345a"). InnerVolumeSpecName "kube-api-access-2wnjh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:41:49.468960 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:49.468924    2558 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8a5d3ce5-76c2-4f07-935f-1bba014a345a-config-file\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:41:49.468960 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:49.468952    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2wnjh\" (UniqueName: \"kubernetes.io/projected/8a5d3ce5-76c2-4f07-935f-1bba014a345a-kube-api-access-2wnjh\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:41:49.589101 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:49.589045    2558 generic.go:356] "Generic (PLEG): container finished" podID="8a5d3ce5-76c2-4f07-935f-1bba014a345a" containerID="4747f8c8d4f674dad6d8e7a589e22f0aa8f4eaa0a2a20552254f7ad615fbbe9f" exitCode=0
Sep 12 00:41:49.589101 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:49.589108    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-hfg9d"
Sep 12 00:41:49.589330 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:49.589124    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-hfg9d" event={"ID":"8a5d3ce5-76c2-4f07-935f-1bba014a345a","Type":"ContainerDied","Data":"4747f8c8d4f674dad6d8e7a589e22f0aa8f4eaa0a2a20552254f7ad615fbbe9f"}
Sep 12 00:41:49.589330 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:49.589184    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-hfg9d" event={"ID":"8a5d3ce5-76c2-4f07-935f-1bba014a345a","Type":"ContainerDied","Data":"f895399862d86f46b6e8266dc981b447e24abe3be67a0c91a169f037bbe9b37c"}
Sep 12 00:41:49.589330 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:49.589214    2558 scope.go:117] "RemoveContainer" containerID="4747f8c8d4f674dad6d8e7a589e22f0aa8f4eaa0a2a20552254f7ad615fbbe9f"
Sep 12 00:41:49.600222 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:49.600163    2558 scope.go:117] "RemoveContainer" containerID="4747f8c8d4f674dad6d8e7a589e22f0aa8f4eaa0a2a20552254f7ad615fbbe9f"
Sep 12 00:41:49.600682 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:41:49.600662    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4747f8c8d4f674dad6d8e7a589e22f0aa8f4eaa0a2a20552254f7ad615fbbe9f\": container with ID starting with 4747f8c8d4f674dad6d8e7a589e22f0aa8f4eaa0a2a20552254f7ad615fbbe9f not found: ID does not exist" containerID="4747f8c8d4f674dad6d8e7a589e22f0aa8f4eaa0a2a20552254f7ad615fbbe9f"
Sep 12 00:41:49.600783 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:49.600691    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4747f8c8d4f674dad6d8e7a589e22f0aa8f4eaa0a2a20552254f7ad615fbbe9f"} err="failed to get container status \"4747f8c8d4f674dad6d8e7a589e22f0aa8f4eaa0a2a20552254f7ad615fbbe9f\": rpc error: code = NotFound desc = could not find container \"4747f8c8d4f674dad6d8e7a589e22f0aa8f4eaa0a2a20552254f7ad615fbbe9f\": container with ID starting with 4747f8c8d4f674dad6d8e7a589e22f0aa8f4eaa0a2a20552254f7ad615fbbe9f not found: ID does not exist"
Sep 12 00:41:49.609960 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:49.609937    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hfg9d"]
Sep 12 00:41:49.612972 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:49.612951    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hfg9d"]
Sep 12 00:41:49.737785 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:49.737753    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a5d3ce5-76c2-4f07-935f-1bba014a345a" path="/var/lib/kubelet/pods/8a5d3ce5-76c2-4f07-935f-1bba014a345a/volumes"
Sep 12 00:41:52.285152 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:52.285125    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:41:52.382985 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:52.382951    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:41:52.720448 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:52.720419    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-5l4sh"]
Sep 12 00:41:52.720781 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:52.720766    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a5d3ce5-76c2-4f07-935f-1bba014a345a" containerName="limitador"
Sep 12 00:41:52.720831 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:52.720783    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a5d3ce5-76c2-4f07-935f-1bba014a345a" containerName="limitador"
Sep 12 00:41:52.720870 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:52.720856    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a5d3ce5-76c2-4f07-935f-1bba014a345a" containerName="limitador"
Sep 12 00:41:52.734044 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:52.734013    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-5l4sh"]
Sep 12 00:41:52.734223 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:52.734152    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5l4sh"
Sep 12 00:41:52.736764 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:52.736598    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-txfp8\""
Sep 12 00:41:52.736764 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:52.736639    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 12 00:41:52.736764 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:52.736707    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 12 00:41:52.737726 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:52.737704    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 12 00:41:52.752037 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:52.752014    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:41:52.898910 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:52.898874    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlbc9\" (UniqueName: \"kubernetes.io/projected/c9c1d977-2a40-4397-bda4-813db9549e9f-kube-api-access-rlbc9\") pod \"postgres-69fb4594cf-5l4sh\" (UID: \"c9c1d977-2a40-4397-bda4-813db9549e9f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5l4sh"
Sep 12 00:41:52.899095 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:52.898939    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c9c1d977-2a40-4397-bda4-813db9549e9f-data\") pod \"postgres-69fb4594cf-5l4sh\" (UID: \"c9c1d977-2a40-4397-bda4-813db9549e9f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5l4sh"
Sep 12 00:41:53.000211 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:53.000112    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlbc9\" (UniqueName: \"kubernetes.io/projected/c9c1d977-2a40-4397-bda4-813db9549e9f-kube-api-access-rlbc9\") pod \"postgres-69fb4594cf-5l4sh\" (UID: \"c9c1d977-2a40-4397-bda4-813db9549e9f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5l4sh"
Sep 12 00:41:53.000211 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:53.000163    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c9c1d977-2a40-4397-bda4-813db9549e9f-data\") pod \"postgres-69fb4594cf-5l4sh\" (UID: \"c9c1d977-2a40-4397-bda4-813db9549e9f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5l4sh"
Sep 12 00:41:53.000614 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:53.000594    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c9c1d977-2a40-4397-bda4-813db9549e9f-data\") pod \"postgres-69fb4594cf-5l4sh\" (UID: \"c9c1d977-2a40-4397-bda4-813db9549e9f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5l4sh"
Sep 12 00:41:53.008870 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:53.008851    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlbc9\" (UniqueName: \"kubernetes.io/projected/c9c1d977-2a40-4397-bda4-813db9549e9f-kube-api-access-rlbc9\") pod \"postgres-69fb4594cf-5l4sh\" (UID: \"c9c1d977-2a40-4397-bda4-813db9549e9f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5l4sh"
Sep 12 00:41:53.047481 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:53.047453    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5l4sh"
Sep 12 00:41:53.320212 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:53.320187    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-5l4sh"]
Sep 12 00:41:53.322593 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:41:53.322566    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9c1d977_2a40_4397_bda4_813db9549e9f.slice/crio-de6328ba1ecbf69e263ae2f41408633a33f27ea235d37e103bb9292026c08afe WatchSource:0}: Error finding container de6328ba1ecbf69e263ae2f41408633a33f27ea235d37e103bb9292026c08afe: Status 404 returned error can't find the container with id de6328ba1ecbf69e263ae2f41408633a33f27ea235d37e103bb9292026c08afe
Sep 12 00:41:53.607916 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:53.607835    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5l4sh" event={"ID":"c9c1d977-2a40-4397-bda4-813db9549e9f","Type":"ContainerStarted","Data":"de6328ba1ecbf69e263ae2f41408633a33f27ea235d37e103bb9292026c08afe"}
Sep 12 00:41:58.634860 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:58.634828    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5l4sh" event={"ID":"c9c1d977-2a40-4397-bda4-813db9549e9f","Type":"ContainerStarted","Data":"a92c86c8d5a4b84a5f1cc8d8e5681c77af962b936534150f1d660b516ebff555"}
Sep 12 00:41:58.635254 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:58.635042    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5l4sh"
Sep 12 00:41:58.650172 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:41:58.650130    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5l4sh" podStartSLOduration=1.863596281 podStartE2EDuration="6.650114005s" totalImagesPullingTime="4.786517724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:41:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:41:58.649288247 +0000 UTC m=+755.323347109" watchObservedRunningTime="2026-09-12 00:41:58.650114005 +0000 UTC m=+755.324172860"
Sep 12 00:42:04.668216 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:04.668186    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5l4sh"
Sep 12 00:42:04.959312 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:04.959282    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:42:05.079359 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:05.079328    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:42:11.583775 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:11.583746    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:42:11.606193 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:11.606165    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-55cc4c7dfc-dn6lt"]
Sep 12 00:42:11.631879 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:11.631849    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55cc4c7dfc-dn6lt"]
Sep 12 00:42:11.632042 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:11.631940    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55cc4c7dfc-dn6lt"
Sep 12 00:42:11.634784 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:11.634765    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 12 00:42:11.634917 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:11.634806    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 12 00:42:11.634917 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:11.634911    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 12 00:42:11.635037 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:11.634939    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5h685\""
Sep 12 00:42:11.652364 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:11.652341    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:42:11.712298 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:11.712274    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:42:11.765472 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:11.765439    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6cf675f9b4-t6jrs"]
Sep 12 00:42:11.767943 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:11.767922    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0fb8f8b8-0988-4290-ac2a-e48e33e619ef-webhook-cert\") pod \"maas-controller-55cc4c7dfc-dn6lt\" (UID: \"0fb8f8b8-0988-4290-ac2a-e48e33e619ef\") " pod="opendatahub/maas-controller-55cc4c7dfc-dn6lt"
Sep 12 00:42:11.768004 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:11.767991    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fb8f8b8-0988-4290-ac2a-e48e33e619ef-metrics-certs\") pod \"maas-controller-55cc4c7dfc-dn6lt\" (UID: \"0fb8f8b8-0988-4290-ac2a-e48e33e619ef\") " pod="opendatahub/maas-controller-55cc4c7dfc-dn6lt"
Sep 12 00:42:11.768040 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:11.768024    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gr4xd\" (UniqueName: \"kubernetes.io/projected/0fb8f8b8-0988-4290-ac2a-e48e33e619ef-kube-api-access-gr4xd\") pod \"maas-controller-55cc4c7dfc-dn6lt\" (UID: \"0fb8f8b8-0988-4290-ac2a-e48e33e619ef\") " pod="opendatahub/maas-controller-55cc4c7dfc-dn6lt"
Sep 12 00:42:11.768937 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:11.768925    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf675f9b4-t6jrs"
Sep 12 00:42:11.781525 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:11.781493    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cf675f9b4-t6jrs"]
Sep 12 00:42:11.782479 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:11.782454    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:42:11.869417 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:11.869338    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tnd8\" (UniqueName: \"kubernetes.io/projected/9216d8e6-c84e-4651-91e6-41da9378e302-kube-api-access-7tnd8\") pod \"maas-controller-6cf675f9b4-t6jrs\" (UID: \"9216d8e6-c84e-4651-91e6-41da9378e302\") " pod="opendatahub/maas-controller-6cf675f9b4-t6jrs"
Sep 12 00:42:11.869417 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:11.869386    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9216d8e6-c84e-4651-91e6-41da9378e302-metrics-certs\") pod \"maas-controller-6cf675f9b4-t6jrs\" (UID: \"9216d8e6-c84e-4651-91e6-41da9378e302\") " pod="opendatahub/maas-controller-6cf675f9b4-t6jrs"
Sep 12 00:42:11.869417 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:11.869404    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9216d8e6-c84e-4651-91e6-41da9378e302-webhook-cert\") pod \"maas-controller-6cf675f9b4-t6jrs\" (UID: \"9216d8e6-c84e-4651-91e6-41da9378e302\") " pod="opendatahub/maas-controller-6cf675f9b4-t6jrs"
Sep 12 00:42:11.869678 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:11.869492    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fb8f8b8-0988-4290-ac2a-e48e33e619ef-metrics-certs\") pod \"maas-controller-55cc4c7dfc-dn6lt\" (UID: \"0fb8f8b8-0988-4290-ac2a-e48e33e619ef\") " pod="opendatahub/maas-controller-55cc4c7dfc-dn6lt"
Sep 12 00:42:11.869678 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:11.869527    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gr4xd\" (UniqueName: \"kubernetes.io/projected/0fb8f8b8-0988-4290-ac2a-e48e33e619ef-kube-api-access-gr4xd\") pod \"maas-controller-55cc4c7dfc-dn6lt\" (UID: \"0fb8f8b8-0988-4290-ac2a-e48e33e619ef\") " pod="opendatahub/maas-controller-55cc4c7dfc-dn6lt"
Sep 12 00:42:11.869678 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:11.869588    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0fb8f8b8-0988-4290-ac2a-e48e33e619ef-webhook-cert\") pod \"maas-controller-55cc4c7dfc-dn6lt\" (UID: \"0fb8f8b8-0988-4290-ac2a-e48e33e619ef\") " pod="opendatahub/maas-controller-55cc4c7dfc-dn6lt"
Sep 12 00:42:11.869815 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:42:11.869707    2558 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 12 00:42:11.869815 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:42:11.869776    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fb8f8b8-0988-4290-ac2a-e48e33e619ef-webhook-cert podName:0fb8f8b8-0988-4290-ac2a-e48e33e619ef nodeName:}" failed. No retries permitted until 2026-09-12 00:42:12.369754751 +0000 UTC m=+769.043813603 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/0fb8f8b8-0988-4290-ac2a-e48e33e619ef-webhook-cert") pod "maas-controller-55cc4c7dfc-dn6lt" (UID: "0fb8f8b8-0988-4290-ac2a-e48e33e619ef") : secret "maas-controller-webhook-cert" not found
Sep 12 00:42:11.871908 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:11.871890    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fb8f8b8-0988-4290-ac2a-e48e33e619ef-metrics-certs\") pod \"maas-controller-55cc4c7dfc-dn6lt\" (UID: \"0fb8f8b8-0988-4290-ac2a-e48e33e619ef\") " pod="opendatahub/maas-controller-55cc4c7dfc-dn6lt"
Sep 12 00:42:11.879905 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:11.879861    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gr4xd\" (UniqueName: \"kubernetes.io/projected/0fb8f8b8-0988-4290-ac2a-e48e33e619ef-kube-api-access-gr4xd\") pod \"maas-controller-55cc4c7dfc-dn6lt\" (UID: \"0fb8f8b8-0988-4290-ac2a-e48e33e619ef\") " pod="opendatahub/maas-controller-55cc4c7dfc-dn6lt"
Sep 12 00:42:11.970382 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:11.970345    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tnd8\" (UniqueName: \"kubernetes.io/projected/9216d8e6-c84e-4651-91e6-41da9378e302-kube-api-access-7tnd8\") pod \"maas-controller-6cf675f9b4-t6jrs\" (UID: \"9216d8e6-c84e-4651-91e6-41da9378e302\") " pod="opendatahub/maas-controller-6cf675f9b4-t6jrs"
Sep 12 00:42:11.970382 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:11.970395    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9216d8e6-c84e-4651-91e6-41da9378e302-metrics-certs\") pod \"maas-controller-6cf675f9b4-t6jrs\" (UID: \"9216d8e6-c84e-4651-91e6-41da9378e302\") " pod="opendatahub/maas-controller-6cf675f9b4-t6jrs"
Sep 12 00:42:11.970617 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:11.970414    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9216d8e6-c84e-4651-91e6-41da9378e302-webhook-cert\") pod \"maas-controller-6cf675f9b4-t6jrs\" (UID: \"9216d8e6-c84e-4651-91e6-41da9378e302\") " pod="opendatahub/maas-controller-6cf675f9b4-t6jrs"
Sep 12 00:42:11.970617 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:42:11.970533    2558 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 12 00:42:11.970617 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:42:11.970598    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9216d8e6-c84e-4651-91e6-41da9378e302-webhook-cert podName:9216d8e6-c84e-4651-91e6-41da9378e302 nodeName:}" failed. No retries permitted until 2026-09-12 00:42:12.470580071 +0000 UTC m=+769.144638908 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/9216d8e6-c84e-4651-91e6-41da9378e302-webhook-cert") pod "maas-controller-6cf675f9b4-t6jrs" (UID: "9216d8e6-c84e-4651-91e6-41da9378e302") : secret "maas-controller-webhook-cert" not found
Sep 12 00:42:11.973064 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:11.973035    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9216d8e6-c84e-4651-91e6-41da9378e302-metrics-certs\") pod \"maas-controller-6cf675f9b4-t6jrs\" (UID: \"9216d8e6-c84e-4651-91e6-41da9378e302\") " pod="opendatahub/maas-controller-6cf675f9b4-t6jrs"
Sep 12 00:42:11.982149 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:11.982118    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tnd8\" (UniqueName: \"kubernetes.io/projected/9216d8e6-c84e-4651-91e6-41da9378e302-kube-api-access-7tnd8\") pod \"maas-controller-6cf675f9b4-t6jrs\" (UID: \"9216d8e6-c84e-4651-91e6-41da9378e302\") " pod="opendatahub/maas-controller-6cf675f9b4-t6jrs"
Sep 12 00:42:12.087965 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:12.087930    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:42:12.375003 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:12.374961    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0fb8f8b8-0988-4290-ac2a-e48e33e619ef-webhook-cert\") pod \"maas-controller-55cc4c7dfc-dn6lt\" (UID: \"0fb8f8b8-0988-4290-ac2a-e48e33e619ef\") " pod="opendatahub/maas-controller-55cc4c7dfc-dn6lt"
Sep 12 00:42:12.377548 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:12.377531    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0fb8f8b8-0988-4290-ac2a-e48e33e619ef-webhook-cert\") pod \"maas-controller-55cc4c7dfc-dn6lt\" (UID: \"0fb8f8b8-0988-4290-ac2a-e48e33e619ef\") " pod="opendatahub/maas-controller-55cc4c7dfc-dn6lt"
Sep 12 00:42:12.475985 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:12.475943    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9216d8e6-c84e-4651-91e6-41da9378e302-webhook-cert\") pod \"maas-controller-6cf675f9b4-t6jrs\" (UID: \"9216d8e6-c84e-4651-91e6-41da9378e302\") " pod="opendatahub/maas-controller-6cf675f9b4-t6jrs"
Sep 12 00:42:12.478442 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:12.478412    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9216d8e6-c84e-4651-91e6-41da9378e302-webhook-cert\") pod \"maas-controller-6cf675f9b4-t6jrs\" (UID: \"9216d8e6-c84e-4651-91e6-41da9378e302\") " pod="opendatahub/maas-controller-6cf675f9b4-t6jrs"
Sep 12 00:42:12.541492 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:12.541448    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55cc4c7dfc-dn6lt"
Sep 12 00:42:12.685841 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:12.685812    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf675f9b4-t6jrs"
Sep 12 00:42:12.810663 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:12.810639    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55cc4c7dfc-dn6lt"]
Sep 12 00:42:12.812789 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:42:12.812766    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fb8f8b8_0988_4290_ac2a_e48e33e619ef.slice/crio-e2243dbe1a8a1f7762a809914f8ff2be078c109f6f455d44f9f864f062748a15 WatchSource:0}: Error finding container e2243dbe1a8a1f7762a809914f8ff2be078c109f6f455d44f9f864f062748a15: Status 404 returned error can't find the container with id e2243dbe1a8a1f7762a809914f8ff2be078c109f6f455d44f9f864f062748a15
Sep 12 00:42:12.957772 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:12.957749    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cf675f9b4-t6jrs"]
Sep 12 00:42:12.959715 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:42:12.959687    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9216d8e6_c84e_4651_91e6_41da9378e302.slice/crio-fae4929eb8afa7187433c2a67ea5782a94dfe402afec2efb811a89b993cdd030 WatchSource:0}: Error finding container fae4929eb8afa7187433c2a67ea5782a94dfe402afec2efb811a89b993cdd030: Status 404 returned error can't find the container with id fae4929eb8afa7187433c2a67ea5782a94dfe402afec2efb811a89b993cdd030
Sep 12 00:42:13.706445 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:13.706363    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf675f9b4-t6jrs" event={"ID":"9216d8e6-c84e-4651-91e6-41da9378e302","Type":"ContainerStarted","Data":"fae4929eb8afa7187433c2a67ea5782a94dfe402afec2efb811a89b993cdd030"}
Sep 12 00:42:13.709688 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:13.709606    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55cc4c7dfc-dn6lt" event={"ID":"0fb8f8b8-0988-4290-ac2a-e48e33e619ef","Type":"ContainerStarted","Data":"e2243dbe1a8a1f7762a809914f8ff2be078c109f6f455d44f9f864f062748a15"}
Sep 12 00:42:15.696403 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:15.696356    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:42:15.720403 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:15.720364    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf675f9b4-t6jrs" event={"ID":"9216d8e6-c84e-4651-91e6-41da9378e302","Type":"ContainerStarted","Data":"cf8773af148e0da6b496531270d792f55556031ae4a17094ef59e512720dde43"}
Sep 12 00:42:15.720587 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:15.720484    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6cf675f9b4-t6jrs"
Sep 12 00:42:15.721947 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:15.721928    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55cc4c7dfc-dn6lt" event={"ID":"0fb8f8b8-0988-4290-ac2a-e48e33e619ef","Type":"ContainerStarted","Data":"b8e40fd1f5ea226a8e6d16ec83be8cc223e6de3c13ef40c5889487959e142787"}
Sep 12 00:42:15.722071 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:15.722032    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-55cc4c7dfc-dn6lt"
Sep 12 00:42:15.737939 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:15.737846    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6cf675f9b4-t6jrs" podStartSLOduration=2.195596651 podStartE2EDuration="4.737830767s" totalImagesPullingTime="2.542234116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:42:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:42:15.737076754 +0000 UTC m=+772.411135691" watchObservedRunningTime="2026-09-12 00:42:15.737830767 +0000 UTC m=+772.411889624"
Sep 12 00:42:15.753604 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:15.753551    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-55cc4c7dfc-dn6lt" podStartSLOduration=2.074081305 podStartE2EDuration="4.75353274s" totalImagesPullingTime="2.679451435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:42:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:42:15.753161899 +0000 UTC m=+772.427220763" watchObservedRunningTime="2026-09-12 00:42:15.75353274 +0000 UTC m=+772.427591599"
Sep 12 00:42:15.797220 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:15.797190    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:42:16.357481 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:16.357449    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:42:26.728805 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:26.728777    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-55cc4c7dfc-dn6lt"
Sep 12 00:42:26.729197 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:26.728844    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6cf675f9b4-t6jrs"
Sep 12 00:42:26.792879 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:26.792847    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-55cc4c7dfc-dn6lt"]
Sep 12 00:42:26.793147 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:26.793124    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-55cc4c7dfc-dn6lt" podUID="0fb8f8b8-0988-4290-ac2a-e48e33e619ef" containerName="manager" containerID="cri-o://b8e40fd1f5ea226a8e6d16ec83be8cc223e6de3c13ef40c5889487959e142787" gracePeriod=30
Sep 12 00:42:26.803982 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:26.803947    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:42:27.038014 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:27.037990    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55cc4c7dfc-dn6lt"
Sep 12 00:42:27.212372 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:27.212338    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0fb8f8b8-0988-4290-ac2a-e48e33e619ef-webhook-cert\") pod \"0fb8f8b8-0988-4290-ac2a-e48e33e619ef\" (UID: \"0fb8f8b8-0988-4290-ac2a-e48e33e619ef\") "
Sep 12 00:42:27.212527 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:27.212426    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fb8f8b8-0988-4290-ac2a-e48e33e619ef-metrics-certs\") pod \"0fb8f8b8-0988-4290-ac2a-e48e33e619ef\" (UID: \"0fb8f8b8-0988-4290-ac2a-e48e33e619ef\") "
Sep 12 00:42:27.212527 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:27.212448    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gr4xd\" (UniqueName: \"kubernetes.io/projected/0fb8f8b8-0988-4290-ac2a-e48e33e619ef-kube-api-access-gr4xd\") pod \"0fb8f8b8-0988-4290-ac2a-e48e33e619ef\" (UID: \"0fb8f8b8-0988-4290-ac2a-e48e33e619ef\") "
Sep 12 00:42:27.214644 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:27.214614    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0fb8f8b8-0988-4290-ac2a-e48e33e619ef-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "0fb8f8b8-0988-4290-ac2a-e48e33e619ef" (UID: "0fb8f8b8-0988-4290-ac2a-e48e33e619ef"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:42:27.214791 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:27.214677    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0fb8f8b8-0988-4290-ac2a-e48e33e619ef-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0fb8f8b8-0988-4290-ac2a-e48e33e619ef" (UID: "0fb8f8b8-0988-4290-ac2a-e48e33e619ef"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:42:27.214791 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:27.214678    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fb8f8b8-0988-4290-ac2a-e48e33e619ef-kube-api-access-gr4xd" (OuterVolumeSpecName: "kube-api-access-gr4xd") pod "0fb8f8b8-0988-4290-ac2a-e48e33e619ef" (UID: "0fb8f8b8-0988-4290-ac2a-e48e33e619ef"). InnerVolumeSpecName "kube-api-access-gr4xd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:42:27.313939 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:27.313841    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fb8f8b8-0988-4290-ac2a-e48e33e619ef-metrics-certs\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:42:27.313939 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:27.313876    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gr4xd\" (UniqueName: \"kubernetes.io/projected/0fb8f8b8-0988-4290-ac2a-e48e33e619ef-kube-api-access-gr4xd\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:42:27.313939 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:27.313891    2558 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0fb8f8b8-0988-4290-ac2a-e48e33e619ef-webhook-cert\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:42:27.773010 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:27.772978    2558 generic.go:356] "Generic (PLEG): container finished" podID="0fb8f8b8-0988-4290-ac2a-e48e33e619ef" containerID="b8e40fd1f5ea226a8e6d16ec83be8cc223e6de3c13ef40c5889487959e142787" exitCode=0
Sep 12 00:42:27.773430 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:27.773022    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55cc4c7dfc-dn6lt"
Sep 12 00:42:27.773430 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:27.773063    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55cc4c7dfc-dn6lt" event={"ID":"0fb8f8b8-0988-4290-ac2a-e48e33e619ef","Type":"ContainerDied","Data":"b8e40fd1f5ea226a8e6d16ec83be8cc223e6de3c13ef40c5889487959e142787"}
Sep 12 00:42:27.773430 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:27.773131    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55cc4c7dfc-dn6lt" event={"ID":"0fb8f8b8-0988-4290-ac2a-e48e33e619ef","Type":"ContainerDied","Data":"e2243dbe1a8a1f7762a809914f8ff2be078c109f6f455d44f9f864f062748a15"}
Sep 12 00:42:27.773430 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:27.773154    2558 scope.go:117] "RemoveContainer" containerID="b8e40fd1f5ea226a8e6d16ec83be8cc223e6de3c13ef40c5889487959e142787"
Sep 12 00:42:27.781633 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:27.781617    2558 scope.go:117] "RemoveContainer" containerID="b8e40fd1f5ea226a8e6d16ec83be8cc223e6de3c13ef40c5889487959e142787"
Sep 12 00:42:27.781874 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:42:27.781858    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8e40fd1f5ea226a8e6d16ec83be8cc223e6de3c13ef40c5889487959e142787\": container with ID starting with b8e40fd1f5ea226a8e6d16ec83be8cc223e6de3c13ef40c5889487959e142787 not found: ID does not exist" containerID="b8e40fd1f5ea226a8e6d16ec83be8cc223e6de3c13ef40c5889487959e142787"
Sep 12 00:42:27.781915 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:27.781882    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8e40fd1f5ea226a8e6d16ec83be8cc223e6de3c13ef40c5889487959e142787"} err="failed to get container status \"b8e40fd1f5ea226a8e6d16ec83be8cc223e6de3c13ef40c5889487959e142787\": rpc error: code = NotFound desc = could not find container \"b8e40fd1f5ea226a8e6d16ec83be8cc223e6de3c13ef40c5889487959e142787\": container with ID starting with b8e40fd1f5ea226a8e6d16ec83be8cc223e6de3c13ef40c5889487959e142787 not found: ID does not exist"
Sep 12 00:42:27.789127 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:27.789104    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-55cc4c7dfc-dn6lt"]
Sep 12 00:42:27.792634 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:27.792613    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-55cc4c7dfc-dn6lt"]
Sep 12 00:42:29.737099 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:29.737046    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0fb8f8b8-0988-4290-ac2a-e48e33e619ef" path="/var/lib/kubelet/pods/0fb8f8b8-0988-4290-ac2a-e48e33e619ef/volumes"
Sep 12 00:42:47.199527 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:47.199439    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:42:47.299549 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:47.299488    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:42:48.668904 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:48.668872    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:42:48.772261 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:48.772227    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:42:48.927689 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:48.927604    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:42:48.977368 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:48.977335    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-84484d59ff-fqvpv"]
Sep 12 00:42:48.977850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:48.977827    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fb8f8b8-0988-4290-ac2a-e48e33e619ef" containerName="manager"
Sep 12 00:42:48.978018 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:48.978007    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fb8f8b8-0988-4290-ac2a-e48e33e619ef" containerName="manager"
Sep 12 00:42:48.978228 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:48.978210    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0fb8f8b8-0988-4290-ac2a-e48e33e619ef" containerName="manager"
Sep 12 00:42:48.984220 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:48.984201    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-84484d59ff-fqvpv"
Sep 12 00:42:48.986847 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:48.986829    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 12 00:42:48.987883 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:48.987864    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 12 00:42:48.987944 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:48.987881    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9r9zt\""
Sep 12 00:42:48.992027 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:48.992011    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-84484d59ff-fqvpv"]
Sep 12 00:42:49.097019 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:49.096983    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cf21cd5-aa7b-47a8-b2da-137f8b53f373-openshift-service-ca\") pod \"maas-api-84484d59ff-fqvpv\" (UID: \"0cf21cd5-aa7b-47a8-b2da-137f8b53f373\") " pod="odh-ai-gateway-infra/maas-api-84484d59ff-fqvpv"
Sep 12 00:42:49.097223 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:49.097039    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cf21cd5-aa7b-47a8-b2da-137f8b53f373-maas-api-tls\") pod \"maas-api-84484d59ff-fqvpv\" (UID: \"0cf21cd5-aa7b-47a8-b2da-137f8b53f373\") " pod="odh-ai-gateway-infra/maas-api-84484d59ff-fqvpv"
Sep 12 00:42:49.097223 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:49.097064    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cf21cd5-aa7b-47a8-b2da-137f8b53f373-metrics-certs\") pod \"maas-api-84484d59ff-fqvpv\" (UID: \"0cf21cd5-aa7b-47a8-b2da-137f8b53f373\") " pod="odh-ai-gateway-infra/maas-api-84484d59ff-fqvpv"
Sep 12 00:42:49.097223 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:49.097140    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvfr9\" (UniqueName: \"kubernetes.io/projected/0cf21cd5-aa7b-47a8-b2da-137f8b53f373-kube-api-access-jvfr9\") pod \"maas-api-84484d59ff-fqvpv\" (UID: \"0cf21cd5-aa7b-47a8-b2da-137f8b53f373\") " pod="odh-ai-gateway-infra/maas-api-84484d59ff-fqvpv"
Sep 12 00:42:49.155103 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:49.155054    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:42:49.198409 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:49.198381    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cf21cd5-aa7b-47a8-b2da-137f8b53f373-maas-api-tls\") pod \"maas-api-84484d59ff-fqvpv\" (UID: \"0cf21cd5-aa7b-47a8-b2da-137f8b53f373\") " pod="odh-ai-gateway-infra/maas-api-84484d59ff-fqvpv"
Sep 12 00:42:49.198569 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:49.198411    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cf21cd5-aa7b-47a8-b2da-137f8b53f373-metrics-certs\") pod \"maas-api-84484d59ff-fqvpv\" (UID: \"0cf21cd5-aa7b-47a8-b2da-137f8b53f373\") " pod="odh-ai-gateway-infra/maas-api-84484d59ff-fqvpv"
Sep 12 00:42:49.198569 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:49.198468    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvfr9\" (UniqueName: \"kubernetes.io/projected/0cf21cd5-aa7b-47a8-b2da-137f8b53f373-kube-api-access-jvfr9\") pod \"maas-api-84484d59ff-fqvpv\" (UID: \"0cf21cd5-aa7b-47a8-b2da-137f8b53f373\") " pod="odh-ai-gateway-infra/maas-api-84484d59ff-fqvpv"
Sep 12 00:42:49.198569 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:49.198536    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cf21cd5-aa7b-47a8-b2da-137f8b53f373-openshift-service-ca\") pod \"maas-api-84484d59ff-fqvpv\" (UID: \"0cf21cd5-aa7b-47a8-b2da-137f8b53f373\") " pod="odh-ai-gateway-infra/maas-api-84484d59ff-fqvpv"
Sep 12 00:42:49.199225 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:49.199202    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cf21cd5-aa7b-47a8-b2da-137f8b53f373-openshift-service-ca\") pod \"maas-api-84484d59ff-fqvpv\" (UID: \"0cf21cd5-aa7b-47a8-b2da-137f8b53f373\") " pod="odh-ai-gateway-infra/maas-api-84484d59ff-fqvpv"
Sep 12 00:42:49.201200 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:49.201181    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cf21cd5-aa7b-47a8-b2da-137f8b53f373-metrics-certs\") pod \"maas-api-84484d59ff-fqvpv\" (UID: \"0cf21cd5-aa7b-47a8-b2da-137f8b53f373\") " pod="odh-ai-gateway-infra/maas-api-84484d59ff-fqvpv"
Sep 12 00:42:49.201316 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:49.201220    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cf21cd5-aa7b-47a8-b2da-137f8b53f373-maas-api-tls\") pod \"maas-api-84484d59ff-fqvpv\" (UID: \"0cf21cd5-aa7b-47a8-b2da-137f8b53f373\") " pod="odh-ai-gateway-infra/maas-api-84484d59ff-fqvpv"
Sep 12 00:42:49.206040 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:49.206018    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvfr9\" (UniqueName: \"kubernetes.io/projected/0cf21cd5-aa7b-47a8-b2da-137f8b53f373-kube-api-access-jvfr9\") pod \"maas-api-84484d59ff-fqvpv\" (UID: \"0cf21cd5-aa7b-47a8-b2da-137f8b53f373\") " pod="odh-ai-gateway-infra/maas-api-84484d59ff-fqvpv"
Sep 12 00:42:49.294959 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:49.294922    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-84484d59ff-fqvpv"
Sep 12 00:42:49.322599 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:49.322569    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:42:49.422799 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:49.422759    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:42:49.429733 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:49.429710    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-lxcxb"]
Sep 12 00:42:49.434322 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:49.434307    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lxcxb"
Sep 12 00:42:49.436823 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:49.436804    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 12 00:42:49.436926 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:49.436834    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-jdlqj\""
Sep 12 00:42:49.447553 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:49.447521    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-lxcxb"]
Sep 12 00:42:49.501630 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:49.501546    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c452359-d6ce-4238-9ccd-73d99a638f4d-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-lxcxb\" (UID: \"0c452359-d6ce-4238-9ccd-73d99a638f4d\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lxcxb"
Sep 12 00:42:49.501782 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:49.501667    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfp7p\" (UniqueName: \"kubernetes.io/projected/0c452359-d6ce-4238-9ccd-73d99a638f4d-kube-api-access-kfp7p\") pod \"payload-pre-processing-59fbc47bf8-lxcxb\" (UID: \"0c452359-d6ce-4238-9ccd-73d99a638f4d\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lxcxb"
Sep 12 00:42:49.570374 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:49.570345    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-84484d59ff-fqvpv"]
Sep 12 00:42:49.573189 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:42:49.573151    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cf21cd5_aa7b_47a8_b2da_137f8b53f373.slice/crio-450cafc8d6f4ce3019851d2c5bb15507f3e38ad5822254ea8082a6f2cb8aa7a5 WatchSource:0}: Error finding container 450cafc8d6f4ce3019851d2c5bb15507f3e38ad5822254ea8082a6f2cb8aa7a5: Status 404 returned error can't find the container with id 450cafc8d6f4ce3019851d2c5bb15507f3e38ad5822254ea8082a6f2cb8aa7a5
Sep 12 00:42:49.603078 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:49.603039    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfp7p\" (UniqueName: \"kubernetes.io/projected/0c452359-d6ce-4238-9ccd-73d99a638f4d-kube-api-access-kfp7p\") pod \"payload-pre-processing-59fbc47bf8-lxcxb\" (UID: \"0c452359-d6ce-4238-9ccd-73d99a638f4d\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lxcxb"
Sep 12 00:42:49.603291 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:49.603147    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c452359-d6ce-4238-9ccd-73d99a638f4d-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-lxcxb\" (UID: \"0c452359-d6ce-4238-9ccd-73d99a638f4d\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lxcxb"
Sep 12 00:42:49.603864 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:49.603844    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c452359-d6ce-4238-9ccd-73d99a638f4d-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-lxcxb\" (UID: \"0c452359-d6ce-4238-9ccd-73d99a638f4d\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lxcxb"
Sep 12 00:42:49.613220 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:49.613198    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfp7p\" (UniqueName: \"kubernetes.io/projected/0c452359-d6ce-4238-9ccd-73d99a638f4d-kube-api-access-kfp7p\") pod \"payload-pre-processing-59fbc47bf8-lxcxb\" (UID: \"0c452359-d6ce-4238-9ccd-73d99a638f4d\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lxcxb"
Sep 12 00:42:49.744830 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:49.744798    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lxcxb"
Sep 12 00:42:49.867740 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:49.867700    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-84484d59ff-fqvpv" event={"ID":"0cf21cd5-aa7b-47a8-b2da-137f8b53f373","Type":"ContainerStarted","Data":"450cafc8d6f4ce3019851d2c5bb15507f3e38ad5822254ea8082a6f2cb8aa7a5"}
Sep 12 00:42:50.018847 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:50.018822    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-lxcxb"]
Sep 12 00:42:50.021442 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:42:50.021413    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c452359_d6ce_4238_9ccd_73d99a638f4d.slice/crio-95b44c187c66c95bdeff58fe55dd6d918350e5379e57586abc72eb6b63095c90 WatchSource:0}: Error finding container 95b44c187c66c95bdeff58fe55dd6d918350e5379e57586abc72eb6b63095c90: Status 404 returned error can't find the container with id 95b44c187c66c95bdeff58fe55dd6d918350e5379e57586abc72eb6b63095c90
Sep 12 00:42:50.876109 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:50.876049    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lxcxb" event={"ID":"0c452359-d6ce-4238-9ccd-73d99a638f4d","Type":"ContainerStarted","Data":"95b44c187c66c95bdeff58fe55dd6d918350e5379e57586abc72eb6b63095c90"}
Sep 12 00:42:51.882543 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:51.882500    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-84484d59ff-fqvpv" event={"ID":"0cf21cd5-aa7b-47a8-b2da-137f8b53f373","Type":"ContainerStarted","Data":"c2f64ace921f17cfe4c81354a6c2fc5646e04575674a3e0e89f5f3a7b53c2b0b"}
Sep 12 00:42:51.882979 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:51.882626    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-84484d59ff-fqvpv"
Sep 12 00:42:51.901880 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:51.901830    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-84484d59ff-fqvpv" podStartSLOduration=2.166755057 podStartE2EDuration="3.901813164s" totalImagesPullingTime="1.735058107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:42:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:42:51.899984838 +0000 UTC m=+808.574043744" watchObservedRunningTime="2026-09-12 00:42:51.901813164 +0000 UTC m=+808.575872020"
Sep 12 00:42:52.890598 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:52.890567    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lxcxb" event={"ID":"0c452359-d6ce-4238-9ccd-73d99a638f4d","Type":"ContainerStarted","Data":"80726d6f2461329237aef7368cd7174eac327cfb86a9751ca519538631bee56c"}
Sep 12 00:42:52.891025 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:52.890642    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lxcxb"
Sep 12 00:42:52.907711 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:52.907662    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lxcxb" podStartSLOduration=1.535489422 podStartE2EDuration="3.907644863s" totalImagesPullingTime="2.372155441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:42:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:42:52.906286124 +0000 UTC m=+809.580344995" watchObservedRunningTime="2026-09-12 00:42:52.907644863 +0000 UTC m=+809.581703721"
Sep 12 00:42:57.896405 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:57.896378    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-84484d59ff-fqvpv"
Sep 12 00:42:58.847842 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:58.847801    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:42:58.953544 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:42:58.953506    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:43:03.897964 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:03.897931    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-lxcxb"
Sep 12 00:43:12.628899 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:12.628864    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:43:24.712523 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:24.712487    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-lj2gn"]
Sep 12 00:43:24.717940 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:24.717921    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj2gn"
Sep 12 00:43:24.721662 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:24.721636    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 12 00:43:24.721822 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:24.721739    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-x99zn\""
Sep 12 00:43:24.721822 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:24.721766    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 12 00:43:24.721948 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:24.721703    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 12 00:43:24.724710 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:24.724688    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-lj2gn"]
Sep 12 00:43:24.730829 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:24.730805    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a52bf419-7387-4576-914f-f8696242137a-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj2gn\" (UID: \"a52bf419-7387-4576-914f-f8696242137a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj2gn"
Sep 12 00:43:24.730982 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:24.730840    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a52bf419-7387-4576-914f-f8696242137a-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj2gn\" (UID: \"a52bf419-7387-4576-914f-f8696242137a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj2gn"
Sep 12 00:43:24.730982 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:24.730947    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a52bf419-7387-4576-914f-f8696242137a-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj2gn\" (UID: \"a52bf419-7387-4576-914f-f8696242137a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj2gn"
Sep 12 00:43:24.730982 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:24.730978    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a52bf419-7387-4576-914f-f8696242137a-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj2gn\" (UID: \"a52bf419-7387-4576-914f-f8696242137a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj2gn"
Sep 12 00:43:24.731215 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:24.731022    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a52bf419-7387-4576-914f-f8696242137a-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj2gn\" (UID: \"a52bf419-7387-4576-914f-f8696242137a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj2gn"
Sep 12 00:43:24.731215 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:24.731047    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fggdp\" (UniqueName: \"kubernetes.io/projected/a52bf419-7387-4576-914f-f8696242137a-kube-api-access-fggdp\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj2gn\" (UID: \"a52bf419-7387-4576-914f-f8696242137a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj2gn"
Sep 12 00:43:24.804867 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:24.804831    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:43:24.831773 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:24.831735    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a52bf419-7387-4576-914f-f8696242137a-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj2gn\" (UID: \"a52bf419-7387-4576-914f-f8696242137a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj2gn"
Sep 12 00:43:24.831773 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:24.831771    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a52bf419-7387-4576-914f-f8696242137a-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj2gn\" (UID: \"a52bf419-7387-4576-914f-f8696242137a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj2gn"
Sep 12 00:43:24.832016 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:24.831795    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a52bf419-7387-4576-914f-f8696242137a-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj2gn\" (UID: \"a52bf419-7387-4576-914f-f8696242137a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj2gn"
Sep 12 00:43:24.832016 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:24.831861    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fggdp\" (UniqueName: \"kubernetes.io/projected/a52bf419-7387-4576-914f-f8696242137a-kube-api-access-fggdp\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj2gn\" (UID: \"a52bf419-7387-4576-914f-f8696242137a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj2gn"
Sep 12 00:43:24.832016 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:24.831922    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a52bf419-7387-4576-914f-f8696242137a-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj2gn\" (UID: \"a52bf419-7387-4576-914f-f8696242137a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj2gn"
Sep 12 00:43:24.832016 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:24.831985    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a52bf419-7387-4576-914f-f8696242137a-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj2gn\" (UID: \"a52bf419-7387-4576-914f-f8696242137a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj2gn"
Sep 12 00:43:24.832249 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:24.832235    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a52bf419-7387-4576-914f-f8696242137a-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj2gn\" (UID: \"a52bf419-7387-4576-914f-f8696242137a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj2gn"
Sep 12 00:43:24.832322 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:24.832273    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a52bf419-7387-4576-914f-f8696242137a-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj2gn\" (UID: \"a52bf419-7387-4576-914f-f8696242137a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj2gn"
Sep 12 00:43:24.832359 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:24.832325    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a52bf419-7387-4576-914f-f8696242137a-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj2gn\" (UID: \"a52bf419-7387-4576-914f-f8696242137a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj2gn"
Sep 12 00:43:24.834354 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:24.834326    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a52bf419-7387-4576-914f-f8696242137a-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj2gn\" (UID: \"a52bf419-7387-4576-914f-f8696242137a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj2gn"
Sep 12 00:43:24.834511 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:24.834493    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a52bf419-7387-4576-914f-f8696242137a-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj2gn\" (UID: \"a52bf419-7387-4576-914f-f8696242137a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj2gn"
Sep 12 00:43:24.839349 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:24.839330    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fggdp\" (UniqueName: \"kubernetes.io/projected/a52bf419-7387-4576-914f-f8696242137a-kube-api-access-fggdp\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lj2gn\" (UID: \"a52bf419-7387-4576-914f-f8696242137a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj2gn"
Sep 12 00:43:25.032423 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:25.032324    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj2gn"
Sep 12 00:43:25.307821 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:25.307566    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-lj2gn"]
Sep 12 00:43:25.309968 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:43:25.309938    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda52bf419_7387_4576_914f_f8696242137a.slice/crio-906ad146a8593ee28f4b78c9640c382121b257d8299bab41256ffac6e21855b7 WatchSource:0}: Error finding container 906ad146a8593ee28f4b78c9640c382121b257d8299bab41256ffac6e21855b7: Status 404 returned error can't find the container with id 906ad146a8593ee28f4b78c9640c382121b257d8299bab41256ffac6e21855b7
Sep 12 00:43:26.053444 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:26.053406    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj2gn" event={"ID":"a52bf419-7387-4576-914f-f8696242137a","Type":"ContainerStarted","Data":"906ad146a8593ee28f4b78c9640c382121b257d8299bab41256ffac6e21855b7"}
Sep 12 00:43:27.060655 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:27.060620    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj2gn" event={"ID":"a52bf419-7387-4576-914f-f8696242137a","Type":"ContainerStarted","Data":"5a855b5bf36c8bc1a822689eab51d1b7b978a66313b477e28d293eaaabfa0bb5"}
Sep 12 00:43:27.079107 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:27.079034    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj2gn" podStartSLOduration=2.063978079 podStartE2EDuration="3.079017949s" totalImagesPullingTime="1.01503987s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:43:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:43:27.077465439 +0000 UTC m=+843.751524299" watchObservedRunningTime="2026-09-12 00:43:27.079017949 +0000 UTC m=+843.753076808"
Sep 12 00:43:35.033140 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:35.033079    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj2gn"
Sep 12 00:43:35.033684 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:35.033173    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj2gn"
Sep 12 00:43:35.046013 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:35.045984    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj2gn"
Sep 12 00:43:35.107223 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:35.107195    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lj2gn"
Sep 12 00:43:35.888708 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:35.888670    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:43:35.988751 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:35.988718    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:43:47.849069 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:43:47.849020    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:44:08.389677 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:08.389639    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:44:08.490217 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:08.490184    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:44:08.507951 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:08.507925    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-lrws2"]
Sep 12 00:44:08.512996 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:08.512980    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lrws2"
Sep 12 00:44:08.515556 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:08.515532    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 12 00:44:08.521792 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:08.521770    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-lrws2"]
Sep 12 00:44:08.530180 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:08.530157    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d4a6f94-092b-4fa0-be27-3748a7fe1658-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lrws2\" (UID: \"0d4a6f94-092b-4fa0-be27-3748a7fe1658\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lrws2"
Sep 12 00:44:08.530309 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:08.530183    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5p5dx\" (UniqueName: \"kubernetes.io/projected/0d4a6f94-092b-4fa0-be27-3748a7fe1658-kube-api-access-5p5dx\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lrws2\" (UID: \"0d4a6f94-092b-4fa0-be27-3748a7fe1658\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lrws2"
Sep 12 00:44:08.530309 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:08.530206    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d4a6f94-092b-4fa0-be27-3748a7fe1658-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lrws2\" (UID: \"0d4a6f94-092b-4fa0-be27-3748a7fe1658\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lrws2"
Sep 12 00:44:08.530309 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:08.530231    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d4a6f94-092b-4fa0-be27-3748a7fe1658-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lrws2\" (UID: \"0d4a6f94-092b-4fa0-be27-3748a7fe1658\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lrws2"
Sep 12 00:44:08.530309 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:08.530293    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d4a6f94-092b-4fa0-be27-3748a7fe1658-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lrws2\" (UID: \"0d4a6f94-092b-4fa0-be27-3748a7fe1658\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lrws2"
Sep 12 00:44:08.530592 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:08.530339    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d4a6f94-092b-4fa0-be27-3748a7fe1658-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lrws2\" (UID: \"0d4a6f94-092b-4fa0-be27-3748a7fe1658\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lrws2"
Sep 12 00:44:08.618383 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:08.618355    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:44:08.631603 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:08.631566    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d4a6f94-092b-4fa0-be27-3748a7fe1658-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lrws2\" (UID: \"0d4a6f94-092b-4fa0-be27-3748a7fe1658\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lrws2"
Sep 12 00:44:08.631767 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:08.631608    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p5dx\" (UniqueName: \"kubernetes.io/projected/0d4a6f94-092b-4fa0-be27-3748a7fe1658-kube-api-access-5p5dx\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lrws2\" (UID: \"0d4a6f94-092b-4fa0-be27-3748a7fe1658\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lrws2"
Sep 12 00:44:08.631767 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:08.631637    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d4a6f94-092b-4fa0-be27-3748a7fe1658-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lrws2\" (UID: \"0d4a6f94-092b-4fa0-be27-3748a7fe1658\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lrws2"
Sep 12 00:44:08.631767 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:08.631678    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d4a6f94-092b-4fa0-be27-3748a7fe1658-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lrws2\" (UID: \"0d4a6f94-092b-4fa0-be27-3748a7fe1658\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lrws2"
Sep 12 00:44:08.631767 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:08.631738    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d4a6f94-092b-4fa0-be27-3748a7fe1658-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lrws2\" (UID: \"0d4a6f94-092b-4fa0-be27-3748a7fe1658\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lrws2"
Sep 12 00:44:08.631985 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:08.631787    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d4a6f94-092b-4fa0-be27-3748a7fe1658-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lrws2\" (UID: \"0d4a6f94-092b-4fa0-be27-3748a7fe1658\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lrws2"
Sep 12 00:44:08.631985 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:08.631950    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d4a6f94-092b-4fa0-be27-3748a7fe1658-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lrws2\" (UID: \"0d4a6f94-092b-4fa0-be27-3748a7fe1658\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lrws2"
Sep 12 00:44:08.632110 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:08.632063    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d4a6f94-092b-4fa0-be27-3748a7fe1658-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lrws2\" (UID: \"0d4a6f94-092b-4fa0-be27-3748a7fe1658\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lrws2"
Sep 12 00:44:08.632110 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:08.632071    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d4a6f94-092b-4fa0-be27-3748a7fe1658-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lrws2\" (UID: \"0d4a6f94-092b-4fa0-be27-3748a7fe1658\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lrws2"
Sep 12 00:44:08.634235 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:08.634206    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d4a6f94-092b-4fa0-be27-3748a7fe1658-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lrws2\" (UID: \"0d4a6f94-092b-4fa0-be27-3748a7fe1658\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lrws2"
Sep 12 00:44:08.634402 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:08.634384    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d4a6f94-092b-4fa0-be27-3748a7fe1658-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lrws2\" (UID: \"0d4a6f94-092b-4fa0-be27-3748a7fe1658\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lrws2"
Sep 12 00:44:08.642053 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:08.642002    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p5dx\" (UniqueName: \"kubernetes.io/projected/0d4a6f94-092b-4fa0-be27-3748a7fe1658-kube-api-access-5p5dx\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lrws2\" (UID: \"0d4a6f94-092b-4fa0-be27-3748a7fe1658\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lrws2"
Sep 12 00:44:08.823598 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:08.823560    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lrws2"
Sep 12 00:44:09.093848 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:09.093825    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-lrws2"]
Sep 12 00:44:09.096242 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:44:09.096213    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d4a6f94_092b_4fa0_be27_3748a7fe1658.slice/crio-f72327e82363d5530a86448ab251f55569e5328e3dfe1d308065c78d987d1204 WatchSource:0}: Error finding container f72327e82363d5530a86448ab251f55569e5328e3dfe1d308065c78d987d1204: Status 404 returned error can't find the container with id f72327e82363d5530a86448ab251f55569e5328e3dfe1d308065c78d987d1204
Sep 12 00:44:09.251485 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:09.251447    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lrws2" event={"ID":"0d4a6f94-092b-4fa0-be27-3748a7fe1658","Type":"ContainerStarted","Data":"f72327e82363d5530a86448ab251f55569e5328e3dfe1d308065c78d987d1204"}
Sep 12 00:44:10.257552 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:10.257510    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lrws2" event={"ID":"0d4a6f94-092b-4fa0-be27-3748a7fe1658","Type":"ContainerStarted","Data":"95ba826a252a1fbc362bf2033e6724c6b898d754f00475cdae924d209901bcbe"}
Sep 12 00:44:10.282295 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:10.282247    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lrws2" podStartSLOduration=2.101409122 podStartE2EDuration="2.282228516s" totalImagesPullingTime="180.819394ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:44:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:44:10.280798365 +0000 UTC m=+886.954857223" watchObservedRunningTime="2026-09-12 00:44:10.282228516 +0000 UTC m=+886.956287376"
Sep 12 00:44:18.824397 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:18.824309    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lrws2"
Sep 12 00:44:18.824397 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:18.824372    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lrws2"
Sep 12 00:44:18.837350 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:18.837321    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lrws2"
Sep 12 00:44:19.311073 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:19.311043    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lrws2"
Sep 12 00:44:23.615519 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:23.615479    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n"]
Sep 12 00:44:23.618997 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:23.618979    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n"
Sep 12 00:44:23.621305 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:23.621286    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 12 00:44:23.627202 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:23.627177    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n"]
Sep 12 00:44:23.669381 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:23.669348    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cw95\" (UniqueName: \"kubernetes.io/projected/c7aa001a-6286-432c-a666-682a4f925a2e-kube-api-access-9cw95\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n\" (UID: \"c7aa001a-6286-432c-a666-682a4f925a2e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n"
Sep 12 00:44:23.669567 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:23.669398    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7aa001a-6286-432c-a666-682a4f925a2e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n\" (UID: \"c7aa001a-6286-432c-a666-682a4f925a2e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n"
Sep 12 00:44:23.669567 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:23.669457    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7aa001a-6286-432c-a666-682a4f925a2e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n\" (UID: \"c7aa001a-6286-432c-a666-682a4f925a2e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n"
Sep 12 00:44:23.669567 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:23.669535    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7aa001a-6286-432c-a666-682a4f925a2e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n\" (UID: \"c7aa001a-6286-432c-a666-682a4f925a2e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n"
Sep 12 00:44:23.669703 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:23.669594    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7aa001a-6286-432c-a666-682a4f925a2e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n\" (UID: \"c7aa001a-6286-432c-a666-682a4f925a2e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n"
Sep 12 00:44:23.669703 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:23.669610    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7aa001a-6286-432c-a666-682a4f925a2e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n\" (UID: \"c7aa001a-6286-432c-a666-682a4f925a2e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n"
Sep 12 00:44:23.753379 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:23.753276    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-czf8h_dd90996a-5428-4ccf-a154-c0aa8def9789/console-operator/1.log"
Sep 12 00:44:23.766530 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:23.753529    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-czf8h_dd90996a-5428-4ccf-a154-c0aa8def9789/console-operator/1.log"
Sep 12 00:44:23.770357 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:23.770333    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7aa001a-6286-432c-a666-682a4f925a2e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n\" (UID: \"c7aa001a-6286-432c-a666-682a4f925a2e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n"
Sep 12 00:44:23.770501 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:23.770385    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7aa001a-6286-432c-a666-682a4f925a2e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n\" (UID: \"c7aa001a-6286-432c-a666-682a4f925a2e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n"
Sep 12 00:44:23.770501 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:23.770456    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7aa001a-6286-432c-a666-682a4f925a2e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n\" (UID: \"c7aa001a-6286-432c-a666-682a4f925a2e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n"
Sep 12 00:44:23.770501 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:23.770479    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7aa001a-6286-432c-a666-682a4f925a2e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n\" (UID: \"c7aa001a-6286-432c-a666-682a4f925a2e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n"
Sep 12 00:44:23.770670 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:23.770534    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cw95\" (UniqueName: \"kubernetes.io/projected/c7aa001a-6286-432c-a666-682a4f925a2e-kube-api-access-9cw95\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n\" (UID: \"c7aa001a-6286-432c-a666-682a4f925a2e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n"
Sep 12 00:44:23.770670 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:23.770581    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7aa001a-6286-432c-a666-682a4f925a2e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n\" (UID: \"c7aa001a-6286-432c-a666-682a4f925a2e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n"
Sep 12 00:44:23.770843 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:23.770822    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7aa001a-6286-432c-a666-682a4f925a2e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n\" (UID: \"c7aa001a-6286-432c-a666-682a4f925a2e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n"
Sep 12 00:44:23.771191 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:23.771167    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7aa001a-6286-432c-a666-682a4f925a2e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n\" (UID: \"c7aa001a-6286-432c-a666-682a4f925a2e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n"
Sep 12 00:44:23.771389 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:23.771367    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7aa001a-6286-432c-a666-682a4f925a2e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n\" (UID: \"c7aa001a-6286-432c-a666-682a4f925a2e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n"
Sep 12 00:44:23.773008 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:23.772984    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7aa001a-6286-432c-a666-682a4f925a2e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n\" (UID: \"c7aa001a-6286-432c-a666-682a4f925a2e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n"
Sep 12 00:44:23.773179 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:23.773162    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7aa001a-6286-432c-a666-682a4f925a2e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n\" (UID: \"c7aa001a-6286-432c-a666-682a4f925a2e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n"
Sep 12 00:44:23.779774 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:23.779749    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cw95\" (UniqueName: \"kubernetes.io/projected/c7aa001a-6286-432c-a666-682a4f925a2e-kube-api-access-9cw95\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n\" (UID: \"c7aa001a-6286-432c-a666-682a4f925a2e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n"
Sep 12 00:44:23.821197 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:23.821168    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:44:23.930407 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:23.930322    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n"
Sep 12 00:44:24.200848 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:24.200823    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n"]
Sep 12 00:44:24.202573 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:44:24.202544    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7aa001a_6286_432c_a666_682a4f925a2e.slice/crio-021366300d018b85b173f107f8980071271634b224a5a80d2f9cc622105b18f1 WatchSource:0}: Error finding container 021366300d018b85b173f107f8980071271634b224a5a80d2f9cc622105b18f1: Status 404 returned error can't find the container with id 021366300d018b85b173f107f8980071271634b224a5a80d2f9cc622105b18f1
Sep 12 00:44:24.323864 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:24.323827    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n" event={"ID":"c7aa001a-6286-432c-a666-682a4f925a2e","Type":"ContainerStarted","Data":"021366300d018b85b173f107f8980071271634b224a5a80d2f9cc622105b18f1"}
Sep 12 00:44:25.330546 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:25.330508    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n" event={"ID":"c7aa001a-6286-432c-a666-682a4f925a2e","Type":"ContainerStarted","Data":"8f7241e761d05baa4a66c435b1d3ed125546975d85b167d04412b9e8ae666365"}
Sep 12 00:44:25.347916 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:25.347859    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n" podStartSLOduration=2.160160678 podStartE2EDuration="2.347843374s" totalImagesPullingTime="187.682696ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:44:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:44:25.34613215 +0000 UTC m=+902.020191010" watchObservedRunningTime="2026-09-12 00:44:25.347843374 +0000 UTC m=+902.021902280"
Sep 12 00:44:33.548194 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:33.548129    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:44:33.930536 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:33.930446    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n"
Sep 12 00:44:33.930536 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:33.930508    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n"
Sep 12 00:44:33.945284 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:33.945261    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n"
Sep 12 00:44:34.390856 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:34.390826    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n"
Sep 12 00:44:46.752677 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:44:46.752646    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:45:49.734000 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:45:49.733891    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:45:49.802190 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:45:49.802160    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:45:50.846979 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:45:50.846940    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:45:50.940984 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:45:50.940951    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:07.085292 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:07.085229    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:07.230574 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:07.230541    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:07.304065 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:07.304032    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:07.520432 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:07.520385    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:07.611290 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:07.611258    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:07.684956 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:07.684927    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:08.054577 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:08.054544    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:08.151133 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:08.151076    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:08.604553 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:08.604518    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:08.713496 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:08.713455    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:09.109679 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:09.109645    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:09.221583 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:09.221544    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7688-e1228d"]
Sep 12 00:46:09.225065 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:09.225040    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7688-e1228d"
Sep 12 00:46:09.227501 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:09.227479    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tqwft\""
Sep 12 00:46:09.232029 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:09.231998    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7688-e1228d"]
Sep 12 00:46:09.282300 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:09.282268    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7m78l\" (UniqueName: \"kubernetes.io/projected/725487e9-bb4f-47fa-843f-49353a093247-kube-api-access-7m78l\") pod \"test-curl-7688-e1228d\" (UID: \"725487e9-bb4f-47fa-843f-49353a093247\") " pod="opendatahub/test-curl-7688-e1228d"
Sep 12 00:46:09.382964 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:09.382862    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7m78l\" (UniqueName: \"kubernetes.io/projected/725487e9-bb4f-47fa-843f-49353a093247-kube-api-access-7m78l\") pod \"test-curl-7688-e1228d\" (UID: \"725487e9-bb4f-47fa-843f-49353a093247\") " pod="opendatahub/test-curl-7688-e1228d"
Sep 12 00:46:09.391063 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:09.391033    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7m78l\" (UniqueName: \"kubernetes.io/projected/725487e9-bb4f-47fa-843f-49353a093247-kube-api-access-7m78l\") pod \"test-curl-7688-e1228d\" (UID: \"725487e9-bb4f-47fa-843f-49353a093247\") " pod="opendatahub/test-curl-7688-e1228d"
Sep 12 00:46:09.536114 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:09.536051    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7688-e1228d"
Sep 12 00:46:09.807416 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:09.807392    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7688-e1228d"]
Sep 12 00:46:09.810318 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:46:09.810292    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod725487e9_bb4f_47fa_843f_49353a093247.slice/crio-c7c8859c13732d107714c9f030699c39ce3d58c505e7aa8e3fad64e0e7b55afd WatchSource:0}: Error finding container c7c8859c13732d107714c9f030699c39ce3d58c505e7aa8e3fad64e0e7b55afd: Status 404 returned error can't find the container with id c7c8859c13732d107714c9f030699c39ce3d58c505e7aa8e3fad64e0e7b55afd
Sep 12 00:46:09.831344 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:09.831318    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7688-e1228d" event={"ID":"725487e9-bb4f-47fa-843f-49353a093247","Type":"ContainerStarted","Data":"c7c8859c13732d107714c9f030699c39ce3d58c505e7aa8e3fad64e0e7b55afd"}
Sep 12 00:46:10.168526 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:10.168399    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:10.259296 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:10.259261    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:11.331334 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:11.331297    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb"]
Sep 12 00:46:11.335850 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:11.335811    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb"
Sep 12 00:46:11.338368 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:11.338339    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-876808-serving-cert\""
Sep 12 00:46:11.348372 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:11.348345    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb"]
Sep 12 00:46:11.368434 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:11.368395    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:11.402585 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:11.402535    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56fdc776-1e9f-4d9d-ac51-536cc781215e-maas-api-tls\") pod \"maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb\" (UID: \"56fdc776-1e9f-4d9d-ac51-536cc781215e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb"
Sep 12 00:46:11.402770 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:11.402634    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56fdc776-1e9f-4d9d-ac51-536cc781215e-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb\" (UID: \"56fdc776-1e9f-4d9d-ac51-536cc781215e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb"
Sep 12 00:46:11.402770 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:11.402694    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56fdc776-1e9f-4d9d-ac51-536cc781215e-metrics-certs\") pod \"maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb\" (UID: \"56fdc776-1e9f-4d9d-ac51-536cc781215e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb"
Sep 12 00:46:11.402770 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:11.402750    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvvp9\" (UniqueName: \"kubernetes.io/projected/56fdc776-1e9f-4d9d-ac51-536cc781215e-kube-api-access-xvvp9\") pod \"maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb\" (UID: \"56fdc776-1e9f-4d9d-ac51-536cc781215e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb"
Sep 12 00:46:11.463960 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:11.463924    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:11.504166 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:11.504120    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvvp9\" (UniqueName: \"kubernetes.io/projected/56fdc776-1e9f-4d9d-ac51-536cc781215e-kube-api-access-xvvp9\") pod \"maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb\" (UID: \"56fdc776-1e9f-4d9d-ac51-536cc781215e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb"
Sep 12 00:46:11.504346 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:11.504187    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56fdc776-1e9f-4d9d-ac51-536cc781215e-maas-api-tls\") pod \"maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb\" (UID: \"56fdc776-1e9f-4d9d-ac51-536cc781215e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb"
Sep 12 00:46:11.504346 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:11.504243    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56fdc776-1e9f-4d9d-ac51-536cc781215e-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb\" (UID: \"56fdc776-1e9f-4d9d-ac51-536cc781215e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb"
Sep 12 00:46:11.504346 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:11.504296    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56fdc776-1e9f-4d9d-ac51-536cc781215e-metrics-certs\") pod \"maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb\" (UID: \"56fdc776-1e9f-4d9d-ac51-536cc781215e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb"
Sep 12 00:46:11.505131 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:11.505106    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56fdc776-1e9f-4d9d-ac51-536cc781215e-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb\" (UID: \"56fdc776-1e9f-4d9d-ac51-536cc781215e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb"
Sep 12 00:46:11.507557 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:11.507533    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56fdc776-1e9f-4d9d-ac51-536cc781215e-maas-api-tls\") pod \"maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb\" (UID: \"56fdc776-1e9f-4d9d-ac51-536cc781215e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb"
Sep 12 00:46:11.507658 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:11.507560    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56fdc776-1e9f-4d9d-ac51-536cc781215e-metrics-certs\") pod \"maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb\" (UID: \"56fdc776-1e9f-4d9d-ac51-536cc781215e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb"
Sep 12 00:46:11.519984 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:11.519952    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvvp9\" (UniqueName: \"kubernetes.io/projected/56fdc776-1e9f-4d9d-ac51-536cc781215e-kube-api-access-xvvp9\") pod \"maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb\" (UID: \"56fdc776-1e9f-4d9d-ac51-536cc781215e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb"
Sep 12 00:46:11.563685 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:11.563643    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:11.652884 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:11.652796    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb"
Sep 12 00:46:12.072347 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:12.072310    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:12.155418 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:12.155388    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:12.195666 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:12.195629    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-876808-7c65579d9d-sqzzc"]
Sep 12 00:46:12.205900 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:12.205874    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-876808-7c65579d9d-sqzzc"
Sep 12 00:46:12.209316 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:12.208528    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-876808\""
Sep 12 00:46:12.209316 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:12.208696    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-876808-dockercfg-cfv4j\""
Sep 12 00:46:12.209316 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:12.209228    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-876808-7c65579d9d-sqzzc"]
Sep 12 00:46:12.304486 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:12.304456    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-876808-6494cfc7b9-xwsw7"]
Sep 12 00:46:12.309770 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:12.309745    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-876808-6494cfc7b9-xwsw7"
Sep 12 00:46:12.312927 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:12.312761    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49a730e9-c6fe-4d17-bd64-233e1d3ba62a-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-876808-7c65579d9d-sqzzc\" (UID: \"49a730e9-c6fe-4d17-bd64-233e1d3ba62a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-876808-7c65579d9d-sqzzc"
Sep 12 00:46:12.312927 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:12.312829    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7sj2\" (UniqueName: \"kubernetes.io/projected/49a730e9-c6fe-4d17-bd64-233e1d3ba62a-kube-api-access-x7sj2\") pod \"payload-pre-processing-e2e-worker-w4-876808-7c65579d9d-sqzzc\" (UID: \"49a730e9-c6fe-4d17-bd64-233e1d3ba62a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-876808-7c65579d9d-sqzzc"
Sep 12 00:46:12.322131 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:12.322078    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-876808-6494cfc7b9-xwsw7"]
Sep 12 00:46:12.413927 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:12.413893    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qzzj\" (UniqueName: \"kubernetes.io/projected/de5f8abf-8e88-43a0-9f50-e1785612895f-kube-api-access-6qzzj\") pod \"payload-processing-e2e-worker-w4-876808-6494cfc7b9-xwsw7\" (UID: \"de5f8abf-8e88-43a0-9f50-e1785612895f\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-876808-6494cfc7b9-xwsw7"
Sep 12 00:46:12.413927 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:12.413941    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de5f8abf-8e88-43a0-9f50-e1785612895f-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-876808-6494cfc7b9-xwsw7\" (UID: \"de5f8abf-8e88-43a0-9f50-e1785612895f\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-876808-6494cfc7b9-xwsw7"
Sep 12 00:46:12.414457 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:12.413966    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7sj2\" (UniqueName: \"kubernetes.io/projected/49a730e9-c6fe-4d17-bd64-233e1d3ba62a-kube-api-access-x7sj2\") pod \"payload-pre-processing-e2e-worker-w4-876808-7c65579d9d-sqzzc\" (UID: \"49a730e9-c6fe-4d17-bd64-233e1d3ba62a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-876808-7c65579d9d-sqzzc"
Sep 12 00:46:12.414457 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:12.414184    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49a730e9-c6fe-4d17-bd64-233e1d3ba62a-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-876808-7c65579d9d-sqzzc\" (UID: \"49a730e9-c6fe-4d17-bd64-233e1d3ba62a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-876808-7c65579d9d-sqzzc"
Sep 12 00:46:12.415136 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:12.415112    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49a730e9-c6fe-4d17-bd64-233e1d3ba62a-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-876808-7c65579d9d-sqzzc\" (UID: \"49a730e9-c6fe-4d17-bd64-233e1d3ba62a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-876808-7c65579d9d-sqzzc"
Sep 12 00:46:12.421708 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:12.421687    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7sj2\" (UniqueName: \"kubernetes.io/projected/49a730e9-c6fe-4d17-bd64-233e1d3ba62a-kube-api-access-x7sj2\") pod \"payload-pre-processing-e2e-worker-w4-876808-7c65579d9d-sqzzc\" (UID: \"49a730e9-c6fe-4d17-bd64-233e1d3ba62a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-876808-7c65579d9d-sqzzc"
Sep 12 00:46:12.515155 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:12.515119    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qzzj\" (UniqueName: \"kubernetes.io/projected/de5f8abf-8e88-43a0-9f50-e1785612895f-kube-api-access-6qzzj\") pod \"payload-processing-e2e-worker-w4-876808-6494cfc7b9-xwsw7\" (UID: \"de5f8abf-8e88-43a0-9f50-e1785612895f\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-876808-6494cfc7b9-xwsw7"
Sep 12 00:46:12.515422 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:12.515191    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de5f8abf-8e88-43a0-9f50-e1785612895f-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-876808-6494cfc7b9-xwsw7\" (UID: \"de5f8abf-8e88-43a0-9f50-e1785612895f\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-876808-6494cfc7b9-xwsw7"
Sep 12 00:46:12.516114 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:12.516077    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de5f8abf-8e88-43a0-9f50-e1785612895f-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-876808-6494cfc7b9-xwsw7\" (UID: \"de5f8abf-8e88-43a0-9f50-e1785612895f\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-876808-6494cfc7b9-xwsw7"
Sep 12 00:46:12.520493 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:12.520474    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-876808-7c65579d9d-sqzzc"
Sep 12 00:46:12.523349 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:12.523327    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qzzj\" (UniqueName: \"kubernetes.io/projected/de5f8abf-8e88-43a0-9f50-e1785612895f-kube-api-access-6qzzj\") pod \"payload-processing-e2e-worker-w4-876808-6494cfc7b9-xwsw7\" (UID: \"de5f8abf-8e88-43a0-9f50-e1785612895f\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-876808-6494cfc7b9-xwsw7"
Sep 12 00:46:12.580406 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:12.580338    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb"]
Sep 12 00:46:12.634078 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:12.634043    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-876808-6494cfc7b9-xwsw7"
Sep 12 00:46:12.794750 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:12.794715    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-876808-7c65579d9d-sqzzc"]
Sep 12 00:46:12.796315 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:46:12.796293    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49a730e9_c6fe_4d17_bd64_233e1d3ba62a.slice/crio-73f5af4190fb9813d09a4071e07ed7ebec568821e88a9ef8176a2a4a4a6afb32 WatchSource:0}: Error finding container 73f5af4190fb9813d09a4071e07ed7ebec568821e88a9ef8176a2a4a4a6afb32: Status 404 returned error can't find the container with id 73f5af4190fb9813d09a4071e07ed7ebec568821e88a9ef8176a2a4a4a6afb32
Sep 12 00:46:12.848305 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:12.848273    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7688-e1228d" event={"ID":"725487e9-bb4f-47fa-843f-49353a093247","Type":"ContainerStarted","Data":"acada43ab7d6cc9f1b322cb31535b1597031c612198ee119315470c843eb3f03"}
Sep 12 00:46:12.849660 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:12.849633    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-876808-7c65579d9d-sqzzc" event={"ID":"49a730e9-c6fe-4d17-bd64-233e1d3ba62a","Type":"ContainerStarted","Data":"73f5af4190fb9813d09a4071e07ed7ebec568821e88a9ef8176a2a4a4a6afb32"}
Sep 12 00:46:12.850768 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:12.850748    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb" event={"ID":"56fdc776-1e9f-4d9d-ac51-536cc781215e","Type":"ContainerStarted","Data":"e148fe4f6215a3689c0804a38236c2343b59d4c4debd933f01a52aaf18042e58"}
Sep 12 00:46:12.864148 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:12.864097    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7688-e1228d" podStartSLOduration=1.305146587 podStartE2EDuration="3.864064498s" totalImagesPullingTime="2.558917911s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:46:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:46:12.862040661 +0000 UTC m=+1009.536099523" watchObservedRunningTime="2026-09-12 00:46:12.864064498 +0000 UTC m=+1009.538123356"
Sep 12 00:46:12.908545 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:12.908511    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-876808-6494cfc7b9-xwsw7"]
Sep 12 00:46:12.912046 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:46:12.912014    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde5f8abf_8e88_43a0_9f50_e1785612895f.slice/crio-a6ec029e7d250c6f3098c00b660fe5067b82df839d794457313aa051d99ea919 WatchSource:0}: Error finding container a6ec029e7d250c6f3098c00b660fe5067b82df839d794457313aa051d99ea919: Status 404 returned error can't find the container with id a6ec029e7d250c6f3098c00b660fe5067b82df839d794457313aa051d99ea919
Sep 12 00:46:13.355458 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:13.355378    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:13.471014 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:13.470975    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:13.477734 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:13.477706    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7688-e1228d"]
Sep 12 00:46:13.480500 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:13.480471    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7688-e1228d"]
Sep 12 00:46:13.567640 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:13.567587    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:13.676521 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:13.676444    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:13.767769 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:13.767736    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:13.858601 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:13.858564    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-876808-6494cfc7b9-xwsw7" event={"ID":"de5f8abf-8e88-43a0-9f50-e1785612895f","Type":"ContainerStarted","Data":"28da67938bcca5f6195c267cc5920aa5841f72a7e98769cf4d000b3966a164bb"}
Sep 12 00:46:13.858808 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:13.858631    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-876808-6494cfc7b9-xwsw7" event={"ID":"de5f8abf-8e88-43a0-9f50-e1785612895f","Type":"ContainerStarted","Data":"a6ec029e7d250c6f3098c00b660fe5067b82df839d794457313aa051d99ea919"}
Sep 12 00:46:13.858808 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:13.858686    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-876808-6494cfc7b9-xwsw7"
Sep 12 00:46:13.860447 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:13.860411    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-876808-7c65579d9d-sqzzc" event={"ID":"49a730e9-c6fe-4d17-bd64-233e1d3ba62a","Type":"ContainerStarted","Data":"2840385bc6469773a23866db5c89163467724ab8dda017eeb021438e0b8dec61"}
Sep 12 00:46:13.860596 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:13.860550    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-876808-7c65579d9d-sqzzc"
Sep 12 00:46:13.862641 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:13.862609    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb" event={"ID":"56fdc776-1e9f-4d9d-ac51-536cc781215e","Type":"ContainerStarted","Data":"5d9a83395a585f6064824d61e28737904d865cc199ac9b03277b01816c7bd12b"}
Sep 12 00:46:13.862843 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:13.862821    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb"
Sep 12 00:46:13.862962 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:46:13.862949    2558 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7688-e1228d" container="test-curl-7688-e1228d"
Sep 12 00:46:13.879695 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:13.879635    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-876808-6494cfc7b9-xwsw7" podStartSLOduration=1.879614295 podStartE2EDuration="1.879614295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:46:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:46:13.877791468 +0000 UTC m=+1010.551850327" watchObservedRunningTime="2026-09-12 00:46:13.879614295 +0000 UTC m=+1010.553673153"
Sep 12 00:46:13.896382 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:13.896315    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb" podStartSLOduration=2.400289513 podStartE2EDuration="2.896291861s" totalImagesPullingTime="496.002348ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:46:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:46:13.894934124 +0000 UTC m=+1010.568992997" watchObservedRunningTime="2026-09-12 00:46:13.896291861 +0000 UTC m=+1010.570350721"
Sep 12 00:46:13.912405 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:13.912326    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-876808-7c65579d9d-sqzzc" podStartSLOduration=1.912305145 podStartE2EDuration="1.912305145s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:46:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:46:13.911756338 +0000 UTC m=+1010.585815201" watchObservedRunningTime="2026-09-12 00:46:13.912305145 +0000 UTC m=+1010.586364007"
Sep 12 00:46:14.598402 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:14.598352    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:14.869042 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:14.868938    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7688-e1228d" podUID="725487e9-bb4f-47fa-843f-49353a093247" containerName="test-curl-7688-e1228d" containerID="cri-o://acada43ab7d6cc9f1b322cb31535b1597031c612198ee119315470c843eb3f03" gracePeriod=2
Sep 12 00:46:15.383463 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:15.383431    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:15.483616 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:15.483586    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:15.666865 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:15.666772    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:15.832956 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:15.832925    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:15.898708 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:15.898677    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:15.951588 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:15.951554    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-4d485e-7c8bb46bcd-s9g6d"]
Sep 12 00:46:15.951955 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:15.951942    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="725487e9-bb4f-47fa-843f-49353a093247" containerName="test-curl-7688-e1228d"
Sep 12 00:46:15.952003 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:15.951957    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="725487e9-bb4f-47fa-843f-49353a093247" containerName="test-curl-7688-e1228d"
Sep 12 00:46:15.952042 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:15.952033    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="725487e9-bb4f-47fa-843f-49353a093247" containerName="test-curl-7688-e1228d"
Sep 12 00:46:15.955303 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:15.955284    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d485e-7c8bb46bcd-s9g6d"
Sep 12 00:46:15.957791 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:15.957771    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-4d485e\""
Sep 12 00:46:15.958398 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:15.958379    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-4d485e-dockercfg-k97bv\""
Sep 12 00:46:15.976660 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:15.976631    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-4d485e-7c8bb46bcd-s9g6d"]
Sep 12 00:46:15.985730 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:15.985704    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-4d485e-65b7fbc5c9-llkv7"]
Sep 12 00:46:15.989326 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:15.989306    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-4d485e-65b7fbc5c9-llkv7"
Sep 12 00:46:15.998514 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:15.998486    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-4d485e-65b7fbc5c9-llkv7"]
Sep 12 00:46:16.050439 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.050403    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/325f7eb9-abb9-4f41-88b9-d488129d4024-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-4d485e-7c8bb46bcd-s9g6d\" (UID: \"325f7eb9-abb9-4f41-88b9-d488129d4024\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d485e-7c8bb46bcd-s9g6d"
Sep 12 00:46:16.050627 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.050475    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whrp8\" (UniqueName: \"kubernetes.io/projected/325f7eb9-abb9-4f41-88b9-d488129d4024-kube-api-access-whrp8\") pod \"payload-pre-processing-e2e-worker-w0-4d485e-7c8bb46bcd-s9g6d\" (UID: \"325f7eb9-abb9-4f41-88b9-d488129d4024\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d485e-7c8bb46bcd-s9g6d"
Sep 12 00:46:16.050627 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.050495    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzx72\" (UniqueName: \"kubernetes.io/projected/55a4d4f6-f077-4ade-9c75-f3c57e8abde4-kube-api-access-fzx72\") pod \"payload-processing-e2e-worker-w0-4d485e-65b7fbc5c9-llkv7\" (UID: \"55a4d4f6-f077-4ade-9c75-f3c57e8abde4\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-4d485e-65b7fbc5c9-llkv7"
Sep 12 00:46:16.050627 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.050529    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55a4d4f6-f077-4ade-9c75-f3c57e8abde4-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-4d485e-65b7fbc5c9-llkv7\" (UID: \"55a4d4f6-f077-4ade-9c75-f3c57e8abde4\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-4d485e-65b7fbc5c9-llkv7"
Sep 12 00:46:16.151043 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.151004    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/325f7eb9-abb9-4f41-88b9-d488129d4024-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-4d485e-7c8bb46bcd-s9g6d\" (UID: \"325f7eb9-abb9-4f41-88b9-d488129d4024\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d485e-7c8bb46bcd-s9g6d"
Sep 12 00:46:16.151256 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.151098    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whrp8\" (UniqueName: \"kubernetes.io/projected/325f7eb9-abb9-4f41-88b9-d488129d4024-kube-api-access-whrp8\") pod \"payload-pre-processing-e2e-worker-w0-4d485e-7c8bb46bcd-s9g6d\" (UID: \"325f7eb9-abb9-4f41-88b9-d488129d4024\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d485e-7c8bb46bcd-s9g6d"
Sep 12 00:46:16.151256 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.151133    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzx72\" (UniqueName: \"kubernetes.io/projected/55a4d4f6-f077-4ade-9c75-f3c57e8abde4-kube-api-access-fzx72\") pod \"payload-processing-e2e-worker-w0-4d485e-65b7fbc5c9-llkv7\" (UID: \"55a4d4f6-f077-4ade-9c75-f3c57e8abde4\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-4d485e-65b7fbc5c9-llkv7"
Sep 12 00:46:16.151256 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.151159    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55a4d4f6-f077-4ade-9c75-f3c57e8abde4-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-4d485e-65b7fbc5c9-llkv7\" (UID: \"55a4d4f6-f077-4ade-9c75-f3c57e8abde4\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-4d485e-65b7fbc5c9-llkv7"
Sep 12 00:46:16.151827 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.151801    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/325f7eb9-abb9-4f41-88b9-d488129d4024-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-4d485e-7c8bb46bcd-s9g6d\" (UID: \"325f7eb9-abb9-4f41-88b9-d488129d4024\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d485e-7c8bb46bcd-s9g6d"
Sep 12 00:46:16.151927 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.151899    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55a4d4f6-f077-4ade-9c75-f3c57e8abde4-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-4d485e-65b7fbc5c9-llkv7\" (UID: \"55a4d4f6-f077-4ade-9c75-f3c57e8abde4\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-4d485e-65b7fbc5c9-llkv7"
Sep 12 00:46:16.159060 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.159031    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzx72\" (UniqueName: \"kubernetes.io/projected/55a4d4f6-f077-4ade-9c75-f3c57e8abde4-kube-api-access-fzx72\") pod \"payload-processing-e2e-worker-w0-4d485e-65b7fbc5c9-llkv7\" (UID: \"55a4d4f6-f077-4ade-9c75-f3c57e8abde4\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-4d485e-65b7fbc5c9-llkv7"
Sep 12 00:46:16.159504 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.159484    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whrp8\" (UniqueName: \"kubernetes.io/projected/325f7eb9-abb9-4f41-88b9-d488129d4024-kube-api-access-whrp8\") pod \"payload-pre-processing-e2e-worker-w0-4d485e-7c8bb46bcd-s9g6d\" (UID: \"325f7eb9-abb9-4f41-88b9-d488129d4024\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d485e-7c8bb46bcd-s9g6d"
Sep 12 00:46:16.266105 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.265992    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d485e-7c8bb46bcd-s9g6d"
Sep 12 00:46:16.280507 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.280479    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:16.300658 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.300628    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-4d485e-65b7fbc5c9-llkv7"
Sep 12 00:46:16.380642 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.380613    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:16.484041 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.484009    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:16.543808 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.543776    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-4d485e-7c8bb46bcd-s9g6d"]
Sep 12 00:46:16.546616 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:46:16.546591    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod325f7eb9_abb9_4f41_88b9_d488129d4024.slice/crio-ba8899d991a9328ea52efbc2de0ee341a3df6932ff21725e6e5f864fd1fc988f WatchSource:0}: Error finding container ba8899d991a9328ea52efbc2de0ee341a3df6932ff21725e6e5f864fd1fc988f: Status 404 returned error can't find the container with id ba8899d991a9328ea52efbc2de0ee341a3df6932ff21725e6e5f864fd1fc988f
Sep 12 00:46:16.592254 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.592229    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-4d485e-65b7fbc5c9-llkv7"]
Sep 12 00:46:16.593896 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:46:16.593872    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55a4d4f6_f077_4ade_9c75_f3c57e8abde4.slice/crio-1e6ec3e8ebf4b5f64b8451438465b6e03182b4a9115c51b525a6a501f2b477cb WatchSource:0}: Error finding container 1e6ec3e8ebf4b5f64b8451438465b6e03182b4a9115c51b525a6a501f2b477cb: Status 404 returned error can't find the container with id 1e6ec3e8ebf4b5f64b8451438465b6e03182b4a9115c51b525a6a501f2b477cb
Sep 12 00:46:16.669361 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.669326    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:16.828509 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.828415    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:16.882581 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.882545    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-4d485e-65b7fbc5c9-llkv7" event={"ID":"55a4d4f6-f077-4ade-9c75-f3c57e8abde4","Type":"ContainerStarted","Data":"d4d581c8685af6a9d912f9567ceea5136d854140d751ac0a1cc9bb6638c0b47c"}
Sep 12 00:46:16.882724 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.882615    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-4d485e-65b7fbc5c9-llkv7" event={"ID":"55a4d4f6-f077-4ade-9c75-f3c57e8abde4","Type":"ContainerStarted","Data":"1e6ec3e8ebf4b5f64b8451438465b6e03182b4a9115c51b525a6a501f2b477cb"}
Sep 12 00:46:16.882795 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.882768    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-4d485e-65b7fbc5c9-llkv7"
Sep 12 00:46:16.884235 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.884212    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d485e-7c8bb46bcd-s9g6d" event={"ID":"325f7eb9-abb9-4f41-88b9-d488129d4024","Type":"ContainerStarted","Data":"51fad3a8d2bbfe2e15485ade9e38243549da315eefbb11fe4ce9e4939cb6f8fe"}
Sep 12 00:46:16.884338 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.884251    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d485e-7c8bb46bcd-s9g6d" event={"ID":"325f7eb9-abb9-4f41-88b9-d488129d4024","Type":"ContainerStarted","Data":"ba8899d991a9328ea52efbc2de0ee341a3df6932ff21725e6e5f864fd1fc988f"}
Sep 12 00:46:16.884338 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.884288    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d485e-7c8bb46bcd-s9g6d"
Sep 12 00:46:16.901854 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.901804    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-4d485e-65b7fbc5c9-llkv7" podStartSLOduration=1.901788549 podStartE2EDuration="1.901788549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:46:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:46:16.900596597 +0000 UTC m=+1013.574655457" watchObservedRunningTime="2026-09-12 00:46:16.901788549 +0000 UTC m=+1013.575847408"
Sep 12 00:46:16.906154 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.906131    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:16.917289 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.917241    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d485e-7c8bb46bcd-s9g6d" podStartSLOduration=1.917224168 podStartE2EDuration="1.917224168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:46:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:46:16.916269054 +0000 UTC m=+1013.590327938" watchObservedRunningTime="2026-09-12 00:46:16.917224168 +0000 UTC m=+1013.591283027"
Sep 12 00:46:16.948005 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.947970    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-81df64-5c8776d946-srz6k"]
Sep 12 00:46:16.952005 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.951980    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-81df64-5c8776d946-srz6k"
Sep 12 00:46:16.954732 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.954706    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-81df64-dockercfg-l5hnt\""
Sep 12 00:46:16.954732 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.954733    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-81df64\""
Sep 12 00:46:16.960038 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:16.959894    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-81df64-5c8776d946-srz6k"]
Sep 12 00:46:17.060900 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:17.060876    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7688-e1228d"
Sep 12 00:46:17.062045 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:17.062022    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d54hh\" (UniqueName: \"kubernetes.io/projected/b5cf8184-7399-4517-a934-cecdb05cb196-kube-api-access-d54hh\") pod \"payload-pre-processing-e2e-inf-a-81df64-5c8776d946-srz6k\" (UID: \"b5cf8184-7399-4517-a934-cecdb05cb196\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-81df64-5c8776d946-srz6k"
Sep 12 00:46:17.062210 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:17.062158    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5cf8184-7399-4517-a934-cecdb05cb196-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-81df64-5c8776d946-srz6k\" (UID: \"b5cf8184-7399-4517-a934-cecdb05cb196\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-81df64-5c8776d946-srz6k"
Sep 12 00:46:17.146393 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:17.146309    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:17.163675 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:17.163636    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7m78l\" (UniqueName: \"kubernetes.io/projected/725487e9-bb4f-47fa-843f-49353a093247-kube-api-access-7m78l\") pod \"725487e9-bb4f-47fa-843f-49353a093247\" (UID: \"725487e9-bb4f-47fa-843f-49353a093247\") "
Sep 12 00:46:17.163835 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:17.163801    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d54hh\" (UniqueName: \"kubernetes.io/projected/b5cf8184-7399-4517-a934-cecdb05cb196-kube-api-access-d54hh\") pod \"payload-pre-processing-e2e-inf-a-81df64-5c8776d946-srz6k\" (UID: \"b5cf8184-7399-4517-a934-cecdb05cb196\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-81df64-5c8776d946-srz6k"
Sep 12 00:46:17.163939 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:17.163923    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5cf8184-7399-4517-a934-cecdb05cb196-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-81df64-5c8776d946-srz6k\" (UID: \"b5cf8184-7399-4517-a934-cecdb05cb196\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-81df64-5c8776d946-srz6k"
Sep 12 00:46:17.164672 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:17.164646    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5cf8184-7399-4517-a934-cecdb05cb196-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-81df64-5c8776d946-srz6k\" (UID: \"b5cf8184-7399-4517-a934-cecdb05cb196\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-81df64-5c8776d946-srz6k"
Sep 12 00:46:17.166412 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:17.166375    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/725487e9-bb4f-47fa-843f-49353a093247-kube-api-access-7m78l" (OuterVolumeSpecName: "kube-api-access-7m78l") pod "725487e9-bb4f-47fa-843f-49353a093247" (UID: "725487e9-bb4f-47fa-843f-49353a093247"). InnerVolumeSpecName "kube-api-access-7m78l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:46:17.172830 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:17.172802    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d54hh\" (UniqueName: \"kubernetes.io/projected/b5cf8184-7399-4517-a934-cecdb05cb196-kube-api-access-d54hh\") pod \"payload-pre-processing-e2e-inf-a-81df64-5c8776d946-srz6k\" (UID: \"b5cf8184-7399-4517-a934-cecdb05cb196\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-81df64-5c8776d946-srz6k"
Sep 12 00:46:17.264736 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:17.264697    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7m78l\" (UniqueName: \"kubernetes.io/projected/725487e9-bb4f-47fa-843f-49353a093247-kube-api-access-7m78l\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:46:17.266586 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:17.266555    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-81df64-5c8776d946-srz6k"
Sep 12 00:46:17.361718 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:17.361681    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:17.467405 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:17.467371    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:17.544583 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:17.544535    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-81df64-5c8776d946-srz6k"]
Sep 12 00:46:17.546414 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:46:17.546383    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5cf8184_7399_4517_a934_cecdb05cb196.slice/crio-b309c5db8b1cc6189fa39094e59bd5d7fdb2c6de7228d9df4f77fb2433964aca WatchSource:0}: Error finding container b309c5db8b1cc6189fa39094e59bd5d7fdb2c6de7228d9df4f77fb2433964aca: Status 404 returned error can't find the container with id b309c5db8b1cc6189fa39094e59bd5d7fdb2c6de7228d9df4f77fb2433964aca
Sep 12 00:46:17.586176 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:17.586142    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:17.739318 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:17.739234    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="725487e9-bb4f-47fa-843f-49353a093247" path="/var/lib/kubelet/pods/725487e9-bb4f-47fa-843f-49353a093247/volumes"
Sep 12 00:46:17.892565 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:17.892529    2558 generic.go:356] "Generic (PLEG): container finished" podID="725487e9-bb4f-47fa-843f-49353a093247" containerID="acada43ab7d6cc9f1b322cb31535b1597031c612198ee119315470c843eb3f03" exitCode=137
Sep 12 00:46:17.892747 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:17.892578    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7688-e1228d"
Sep 12 00:46:17.892747 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:17.892655    2558 scope.go:117] "RemoveContainer" containerID="acada43ab7d6cc9f1b322cb31535b1597031c612198ee119315470c843eb3f03"
Sep 12 00:46:17.896525 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:17.896432    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-81df64-5c8776d946-srz6k" event={"ID":"b5cf8184-7399-4517-a934-cecdb05cb196","Type":"ContainerStarted","Data":"6bf055cb0e6766f45d677cb36107bc4251b3f72ad6e0a7541e01fff4e509a28e"}
Sep 12 00:46:17.896525 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:17.896490    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-81df64-5c8776d946-srz6k" event={"ID":"b5cf8184-7399-4517-a934-cecdb05cb196","Type":"ContainerStarted","Data":"b309c5db8b1cc6189fa39094e59bd5d7fdb2c6de7228d9df4f77fb2433964aca"}
Sep 12 00:46:17.896849 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:17.896831    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-81df64-5c8776d946-srz6k"
Sep 12 00:46:17.902958 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:17.902943    2558 scope.go:117] "RemoveContainer" containerID="acada43ab7d6cc9f1b322cb31535b1597031c612198ee119315470c843eb3f03"
Sep 12 00:46:17.903386 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:46:17.903356    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acada43ab7d6cc9f1b322cb31535b1597031c612198ee119315470c843eb3f03\": container with ID starting with acada43ab7d6cc9f1b322cb31535b1597031c612198ee119315470c843eb3f03 not found: ID does not exist" containerID="acada43ab7d6cc9f1b322cb31535b1597031c612198ee119315470c843eb3f03"
Sep 12 00:46:17.903485 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:17.903393    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acada43ab7d6cc9f1b322cb31535b1597031c612198ee119315470c843eb3f03"} err="failed to get container status \"acada43ab7d6cc9f1b322cb31535b1597031c612198ee119315470c843eb3f03\": rpc error: code = NotFound desc = could not find container \"acada43ab7d6cc9f1b322cb31535b1597031c612198ee119315470c843eb3f03\": container with ID starting with acada43ab7d6cc9f1b322cb31535b1597031c612198ee119315470c843eb3f03 not found: ID does not exist"
Sep 12 00:46:17.931832 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:17.931779    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-81df64-5c8776d946-srz6k" podStartSLOduration=1.93175957 podStartE2EDuration="1.93175957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:46:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:46:17.928711351 +0000 UTC m=+1014.602770209" watchObservedRunningTime="2026-09-12 00:46:17.93175957 +0000 UTC m=+1014.605818430"
Sep 12 00:46:17.998671 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:17.998576    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:18.310328 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:18.310247    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:18.620708 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:18.620618    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:19.170800 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:19.170765    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:19.874855 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:19.874765    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb"
Sep 12 00:46:20.176693 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:20.176598    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:20.353406 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:20.353375    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:20.449354 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:20.449318    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:20.779891 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:20.779803    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:20.877569 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:20.877538    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:21.160357 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:21.160271    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:22.280201 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:22.280171    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:23.089560 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:23.089528    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:23.192158 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:23.192130    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:23.526904 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:23.526867    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:23.629688 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:23.629654    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:23.642001 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:23.641974    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-220bac51-7878986646-zjp7r"]
Sep 12 00:46:23.645562 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:23.645538    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-220bac51-7878986646-zjp7r"
Sep 12 00:46:23.647793 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:23.647774    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-220bac51\""
Sep 12 00:46:23.647890 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:23.647875    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-220bac51-dockercfg-k98kj\""
Sep 12 00:46:23.656142 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:23.656117    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-220bac51-7878986646-zjp7r"]
Sep 12 00:46:23.674751 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:23.674716    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-220bac51-d9bccdcb7-ln7qk"]
Sep 12 00:46:23.678380 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:23.678357    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-220bac51-d9bccdcb7-ln7qk"
Sep 12 00:46:23.693333 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:23.693308    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-220bac51-d9bccdcb7-ln7qk"]
Sep 12 00:46:23.728386 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:23.728353    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxxrh\" (UniqueName: \"kubernetes.io/projected/c437a648-3aa7-464c-bacc-e12fa50f3676-kube-api-access-vxxrh\") pod \"payload-pre-processing-e2e-ait-220bac51-7878986646-zjp7r\" (UID: \"c437a648-3aa7-464c-bacc-e12fa50f3676\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-220bac51-7878986646-zjp7r"
Sep 12 00:46:23.728553 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:23.728469    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c437a648-3aa7-464c-bacc-e12fa50f3676-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-220bac51-7878986646-zjp7r\" (UID: \"c437a648-3aa7-464c-bacc-e12fa50f3676\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-220bac51-7878986646-zjp7r"
Sep 12 00:46:23.829944 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:23.829845    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c437a648-3aa7-464c-bacc-e12fa50f3676-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-220bac51-7878986646-zjp7r\" (UID: \"c437a648-3aa7-464c-bacc-e12fa50f3676\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-220bac51-7878986646-zjp7r"
Sep 12 00:46:23.829944 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:23.829927    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6jkl\" (UniqueName: \"kubernetes.io/projected/9501e6dd-0aaa-494b-b7e5-69c5fb24bc64-kube-api-access-b6jkl\") pod \"payload-processing-e2e-ait-220bac51-d9bccdcb7-ln7qk\" (UID: \"9501e6dd-0aaa-494b-b7e5-69c5fb24bc64\") " pod="openshift-ingress/payload-processing-e2e-ait-220bac51-d9bccdcb7-ln7qk"
Sep 12 00:46:23.830262 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:23.830023    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxxrh\" (UniqueName: \"kubernetes.io/projected/c437a648-3aa7-464c-bacc-e12fa50f3676-kube-api-access-vxxrh\") pod \"payload-pre-processing-e2e-ait-220bac51-7878986646-zjp7r\" (UID: \"c437a648-3aa7-464c-bacc-e12fa50f3676\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-220bac51-7878986646-zjp7r"
Sep 12 00:46:23.830262 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:23.830099    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9501e6dd-0aaa-494b-b7e5-69c5fb24bc64-plugins-config-volume\") pod \"payload-processing-e2e-ait-220bac51-d9bccdcb7-ln7qk\" (UID: \"9501e6dd-0aaa-494b-b7e5-69c5fb24bc64\") " pod="openshift-ingress/payload-processing-e2e-ait-220bac51-d9bccdcb7-ln7qk"
Sep 12 00:46:23.830882 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:23.830855    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c437a648-3aa7-464c-bacc-e12fa50f3676-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-220bac51-7878986646-zjp7r\" (UID: \"c437a648-3aa7-464c-bacc-e12fa50f3676\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-220bac51-7878986646-zjp7r"
Sep 12 00:46:23.837960 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:23.837937    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxxrh\" (UniqueName: \"kubernetes.io/projected/c437a648-3aa7-464c-bacc-e12fa50f3676-kube-api-access-vxxrh\") pod \"payload-pre-processing-e2e-ait-220bac51-7878986646-zjp7r\" (UID: \"c437a648-3aa7-464c-bacc-e12fa50f3676\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-220bac51-7878986646-zjp7r"
Sep 12 00:46:23.930984 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:23.930944    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9501e6dd-0aaa-494b-b7e5-69c5fb24bc64-plugins-config-volume\") pod \"payload-processing-e2e-ait-220bac51-d9bccdcb7-ln7qk\" (UID: \"9501e6dd-0aaa-494b-b7e5-69c5fb24bc64\") " pod="openshift-ingress/payload-processing-e2e-ait-220bac51-d9bccdcb7-ln7qk"
Sep 12 00:46:23.931184 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:23.931075    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6jkl\" (UniqueName: \"kubernetes.io/projected/9501e6dd-0aaa-494b-b7e5-69c5fb24bc64-kube-api-access-b6jkl\") pod \"payload-processing-e2e-ait-220bac51-d9bccdcb7-ln7qk\" (UID: \"9501e6dd-0aaa-494b-b7e5-69c5fb24bc64\") " pod="openshift-ingress/payload-processing-e2e-ait-220bac51-d9bccdcb7-ln7qk"
Sep 12 00:46:23.931770 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:23.931745    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9501e6dd-0aaa-494b-b7e5-69c5fb24bc64-plugins-config-volume\") pod \"payload-processing-e2e-ait-220bac51-d9bccdcb7-ln7qk\" (UID: \"9501e6dd-0aaa-494b-b7e5-69c5fb24bc64\") " pod="openshift-ingress/payload-processing-e2e-ait-220bac51-d9bccdcb7-ln7qk"
Sep 12 00:46:23.939189 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:23.939162    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6jkl\" (UniqueName: \"kubernetes.io/projected/9501e6dd-0aaa-494b-b7e5-69c5fb24bc64-kube-api-access-b6jkl\") pod \"payload-processing-e2e-ait-220bac51-d9bccdcb7-ln7qk\" (UID: \"9501e6dd-0aaa-494b-b7e5-69c5fb24bc64\") " pod="openshift-ingress/payload-processing-e2e-ait-220bac51-d9bccdcb7-ln7qk"
Sep 12 00:46:23.958275 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:23.958245    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-220bac51-7878986646-zjp7r"
Sep 12 00:46:23.966273 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:23.966238    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:23.988122 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:23.988067    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-220bac51-d9bccdcb7-ln7qk"
Sep 12 00:46:24.240802 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:24.240741    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-220bac51-7878986646-zjp7r"]
Sep 12 00:46:24.243571 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:46:24.243544    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc437a648_3aa7_464c_bacc_e12fa50f3676.slice/crio-a8060e78db6a590fb7ab1e896b956022aa5a0581212e3706a2b13335a8f9b511 WatchSource:0}: Error finding container a8060e78db6a590fb7ab1e896b956022aa5a0581212e3706a2b13335a8f9b511: Status 404 returned error can't find the container with id a8060e78db6a590fb7ab1e896b956022aa5a0581212e3706a2b13335a8f9b511
Sep 12 00:46:24.288568 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:24.288541    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-220bac51-d9bccdcb7-ln7qk"]
Sep 12 00:46:24.289957 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:46:24.289930    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9501e6dd_0aaa_494b_b7e5_69c5fb24bc64.slice/crio-f7a5ef49c74e68ef9cf4b7c90ea551897da504a9b79c1c1438341a7b826a2567 WatchSource:0}: Error finding container f7a5ef49c74e68ef9cf4b7c90ea551897da504a9b79c1c1438341a7b826a2567: Status 404 returned error can't find the container with id f7a5ef49c74e68ef9cf4b7c90ea551897da504a9b79c1c1438341a7b826a2567
Sep 12 00:46:24.476012 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:24.475981    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:24.759037 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:24.759006    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm"]
Sep 12 00:46:24.763657 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:24.763636    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm"
Sep 12 00:46:24.766142 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:24.766118    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-4b6ed7-serving-cert\""
Sep 12 00:46:24.776552 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:24.776525    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm"]
Sep 12 00:46:24.789311 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:24.789256    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:24.839261 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:24.839227    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c0d2758-05db-4945-b531-73c1850e4b2c-openshift-service-ca\") pod \"maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm\" (UID: \"7c0d2758-05db-4945-b531-73c1850e4b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm"
Sep 12 00:46:24.839429 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:24.839301    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c0d2758-05db-4945-b531-73c1850e4b2c-maas-api-tls\") pod \"maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm\" (UID: \"7c0d2758-05db-4945-b531-73c1850e4b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm"
Sep 12 00:46:24.839429 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:24.839399    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c0d2758-05db-4945-b531-73c1850e4b2c-metrics-certs\") pod \"maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm\" (UID: \"7c0d2758-05db-4945-b531-73c1850e4b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm"
Sep 12 00:46:24.839565 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:24.839487    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9c8w\" (UniqueName: \"kubernetes.io/projected/7c0d2758-05db-4945-b531-73c1850e4b2c-kube-api-access-r9c8w\") pod \"maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm\" (UID: \"7c0d2758-05db-4945-b531-73c1850e4b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm"
Sep 12 00:46:24.848822 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:24.848791    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:24.871238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:24.871213    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-876808-7c65579d9d-sqzzc"
Sep 12 00:46:24.871662 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:24.871636    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-876808-6494cfc7b9-xwsw7"
Sep 12 00:46:24.939163 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:24.939124    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-220bac51-d9bccdcb7-ln7qk" event={"ID":"9501e6dd-0aaa-494b-b7e5-69c5fb24bc64","Type":"ContainerStarted","Data":"466fb32aebece685effe37697fc107345e130837b3bae367c697ad23f1813e61"}
Sep 12 00:46:24.939348 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:24.939188    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-220bac51-d9bccdcb7-ln7qk" event={"ID":"9501e6dd-0aaa-494b-b7e5-69c5fb24bc64","Type":"ContainerStarted","Data":"f7a5ef49c74e68ef9cf4b7c90ea551897da504a9b79c1c1438341a7b826a2567"}
Sep 12 00:46:24.939348 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:24.939276    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-220bac51-d9bccdcb7-ln7qk"
Sep 12 00:46:24.939982 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:24.939957    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c0d2758-05db-4945-b531-73c1850e4b2c-maas-api-tls\") pod \"maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm\" (UID: \"7c0d2758-05db-4945-b531-73c1850e4b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm"
Sep 12 00:46:24.940143 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:24.940008    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c0d2758-05db-4945-b531-73c1850e4b2c-metrics-certs\") pod \"maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm\" (UID: \"7c0d2758-05db-4945-b531-73c1850e4b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm"
Sep 12 00:46:24.940143 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:24.940063    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9c8w\" (UniqueName: \"kubernetes.io/projected/7c0d2758-05db-4945-b531-73c1850e4b2c-kube-api-access-r9c8w\") pod \"maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm\" (UID: \"7c0d2758-05db-4945-b531-73c1850e4b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm"
Sep 12 00:46:24.940143 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:46:24.940124    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-4b6ed7-serving-cert: secret "maas-api-e2e-autores-4b6ed7-serving-cert" not found
Sep 12 00:46:24.940281 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:24.940157    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c0d2758-05db-4945-b531-73c1850e4b2c-openshift-service-ca\") pod \"maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm\" (UID: \"7c0d2758-05db-4945-b531-73c1850e4b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm"
Sep 12 00:46:24.940281 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:46:24.940179    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c0d2758-05db-4945-b531-73c1850e4b2c-maas-api-tls podName:7c0d2758-05db-4945-b531-73c1850e4b2c nodeName:}" failed. No retries permitted until 2026-09-12 00:46:25.440162308 +0000 UTC m=+1022.114221168 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7c0d2758-05db-4945-b531-73c1850e4b2c-maas-api-tls") pod "maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm" (UID: "7c0d2758-05db-4945-b531-73c1850e4b2c") : secret "maas-api-e2e-autores-4b6ed7-serving-cert" not found
Sep 12 00:46:24.940864 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:24.940841    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c0d2758-05db-4945-b531-73c1850e4b2c-openshift-service-ca\") pod \"maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm\" (UID: \"7c0d2758-05db-4945-b531-73c1850e4b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm"
Sep 12 00:46:24.941291 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:24.941265    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-220bac51-7878986646-zjp7r" event={"ID":"c437a648-3aa7-464c-bacc-e12fa50f3676","Type":"ContainerStarted","Data":"7c8ce5a19c61b76303ecf94da1bf2c2cc1482f7ae5a853de5274b242fae14d5e"}
Sep 12 00:46:24.941400 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:24.941312    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-220bac51-7878986646-zjp7r" event={"ID":"c437a648-3aa7-464c-bacc-e12fa50f3676","Type":"ContainerStarted","Data":"a8060e78db6a590fb7ab1e896b956022aa5a0581212e3706a2b13335a8f9b511"}
Sep 12 00:46:24.941400 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:24.941340    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-220bac51-7878986646-zjp7r"
Sep 12 00:46:24.943213 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:24.943193    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c0d2758-05db-4945-b531-73c1850e4b2c-metrics-certs\") pod \"maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm\" (UID: \"7c0d2758-05db-4945-b531-73c1850e4b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm"
Sep 12 00:46:24.948263 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:24.948238    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:24.948801 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:24.948779    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9c8w\" (UniqueName: \"kubernetes.io/projected/7c0d2758-05db-4945-b531-73c1850e4b2c-kube-api-access-r9c8w\") pod \"maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm\" (UID: \"7c0d2758-05db-4945-b531-73c1850e4b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm"
Sep 12 00:46:24.956540 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:24.956499    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-220bac51-d9bccdcb7-ln7qk" podStartSLOduration=1.9564857 podStartE2EDuration="1.9564857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:46:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:46:24.955251855 +0000 UTC m=+1021.629310714" watchObservedRunningTime="2026-09-12 00:46:24.9564857 +0000 UTC m=+1021.630544559"
Sep 12 00:46:24.975164 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:24.975109    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-220bac51-7878986646-zjp7r" podStartSLOduration=1.975076238 podStartE2EDuration="1.975076238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:46:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:46:24.972313821 +0000 UTC m=+1021.646372681" watchObservedRunningTime="2026-09-12 00:46:24.975076238 +0000 UTC m=+1021.649135098"
Sep 12 00:46:25.310320 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:25.310279    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:25.445139 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:25.445073    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c0d2758-05db-4945-b531-73c1850e4b2c-maas-api-tls\") pod \"maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm\" (UID: \"7c0d2758-05db-4945-b531-73c1850e4b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm"
Sep 12 00:46:25.447776 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:25.447741    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c0d2758-05db-4945-b531-73c1850e4b2c-maas-api-tls\") pod \"maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm\" (UID: \"7c0d2758-05db-4945-b531-73c1850e4b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm"
Sep 12 00:46:25.465282 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:25.465248    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:25.550749 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:25.550718    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:25.689144 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:25.689106    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm"
Sep 12 00:46:25.854242 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:25.854214    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:25.962527 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:25.962446    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:25.982978 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:25.982951    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm"]
Sep 12 00:46:25.984773 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:46:25.984737    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c0d2758_05db_4945_b531_73c1850e4b2c.slice/crio-a33bb1f0ef6669f5d814025ff8737ebb49a852071215459974b9ad28b2ed921b WatchSource:0}: Error finding container a33bb1f0ef6669f5d814025ff8737ebb49a852071215459974b9ad28b2ed921b: Status 404 returned error can't find the container with id a33bb1f0ef6669f5d814025ff8737ebb49a852071215459974b9ad28b2ed921b
Sep 12 00:46:26.063929 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:26.063896    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:26.163296 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:26.163266    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:26.242859 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:26.242789    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:26.473010 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:26.472968    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:26.776134 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:26.776095    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:26.955396 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:26.955363    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm" event={"ID":"7c0d2758-05db-4945-b531-73c1850e4b2c","Type":"ContainerStarted","Data":"2d38bd79b206d7da640ef59b24ae47829c3ae0d0bfa240c53775e4f8e6a6e230"}
Sep 12 00:46:26.955804 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:26.955414    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm" event={"ID":"7c0d2758-05db-4945-b531-73c1850e4b2c","Type":"ContainerStarted","Data":"a33bb1f0ef6669f5d814025ff8737ebb49a852071215459974b9ad28b2ed921b"}
Sep 12 00:46:26.955804 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:26.955485    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm"
Sep 12 00:46:26.977065 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:26.977014    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm" podStartSLOduration=2.641603547 podStartE2EDuration="2.97699963s" totalImagesPullingTime="335.396083ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:46:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:46:26.975918046 +0000 UTC m=+1023.649976908" watchObservedRunningTime="2026-09-12 00:46:26.97699963 +0000 UTC m=+1023.651058545"
Sep 12 00:46:27.476341 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:27.476307    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:27.898343 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:27.898270    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d485e-7c8bb46bcd-s9g6d"
Sep 12 00:46:27.898482 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:27.898346    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-4d485e-65b7fbc5c9-llkv7"
Sep 12 00:46:28.194337 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:28.194306    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:28.870221 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:28.870190    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:28.984141 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:28.984107    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:29.677064 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:29.677027    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:29.912234 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:29.912204    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-81df64-5c8776d946-srz6k"
Sep 12 00:46:30.379571 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:30.379537    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:30.527364 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:30.527327    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:31.254899 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:31.254869    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:31.993769 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:31.993724    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:32.093786 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:32.093742    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:32.836795 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:32.836761    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:32.968254 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:32.968225    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm"
Sep 12 00:46:33.224238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:33.224200    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:33.323553 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:33.323499    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:33.423764 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:33.423730    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:33.557357 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:33.557269    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:33.673210 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:33.673173    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:33.776141 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:33.776102    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:33.870270 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:33.870169    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:33.971030 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:33.970991    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:34.071136 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:34.071071    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:34.323199 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:34.323164    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:34.774224 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:34.774185    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:35.685762 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:35.685734    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:35.951680 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:35.951599    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-220bac51-7878986646-zjp7r"
Sep 12 00:46:35.951829 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:35.951762    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-220bac51-d9bccdcb7-ln7qk"
Sep 12 00:46:36.562240 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:36.562198    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:37.209078 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:37.209040    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:37.479105 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:37.478993    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:37.578914 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:37.578882    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:38.069217 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:38.069177    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:38.895467 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:38.895430    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:39.566478 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:39.566445    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:39.666644 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:39.666608    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:39.861266 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:39.861183    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:40.118475 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:40.118380    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:40.589809 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:40.589764    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:40.687797 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:40.687758    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:41.049570 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:41.049543    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:41.135420 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:41.135385    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:41.136250 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:41.136221    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-bb6e74-6dfbdcd5d-bmst9"]
Sep 12 00:46:41.141246 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:41.141230    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bb6e74-6dfbdcd5d-bmst9"
Sep 12 00:46:41.143702 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:41.143683    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-bb6e74\""
Sep 12 00:46:41.149125 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:41.148796    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-bb6e74-6dfbdcd5d-bmst9"]
Sep 12 00:46:41.196070 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:41.196041    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-bb6e74-59d9668859-c96ng"]
Sep 12 00:46:41.205480 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:41.205448    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-bb6e74-59d9668859-c96ng"
Sep 12 00:46:41.208075 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:41.208046    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hrx7\" (UniqueName: \"kubernetes.io/projected/e38f1710-8367-4e0d-a2d5-e5b277194c2a-kube-api-access-5hrx7\") pod \"payload-pre-processing-e2e-inf-b-bb6e74-6dfbdcd5d-bmst9\" (UID: \"e38f1710-8367-4e0d-a2d5-e5b277194c2a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bb6e74-6dfbdcd5d-bmst9"
Sep 12 00:46:41.208268 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:41.208250    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-bb6e74-dockercfg-hf8p2\""
Sep 12 00:46:41.208340 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:41.208100    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-bb6e74-59d9668859-c96ng"]
Sep 12 00:46:41.208340 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:41.208320    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e38f1710-8367-4e0d-a2d5-e5b277194c2a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-bb6e74-6dfbdcd5d-bmst9\" (UID: \"e38f1710-8367-4e0d-a2d5-e5b277194c2a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bb6e74-6dfbdcd5d-bmst9"
Sep 12 00:46:41.309171 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:41.309057    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e38f1710-8367-4e0d-a2d5-e5b277194c2a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-bb6e74-6dfbdcd5d-bmst9\" (UID: \"e38f1710-8367-4e0d-a2d5-e5b277194c2a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bb6e74-6dfbdcd5d-bmst9"
Sep 12 00:46:41.309344 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:41.309240    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/975e8170-7e78-40dd-a439-a24bd4a50165-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-bb6e74-59d9668859-c96ng\" (UID: \"975e8170-7e78-40dd-a439-a24bd4a50165\") " pod="openshift-ingress/payload-processing-e2e-inf-b-bb6e74-59d9668859-c96ng"
Sep 12 00:46:41.309344 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:41.309270    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgjwc\" (UniqueName: \"kubernetes.io/projected/975e8170-7e78-40dd-a439-a24bd4a50165-kube-api-access-kgjwc\") pod \"payload-processing-e2e-inf-b-bb6e74-59d9668859-c96ng\" (UID: \"975e8170-7e78-40dd-a439-a24bd4a50165\") " pod="openshift-ingress/payload-processing-e2e-inf-b-bb6e74-59d9668859-c96ng"
Sep 12 00:46:41.309344 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:41.309311    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hrx7\" (UniqueName: \"kubernetes.io/projected/e38f1710-8367-4e0d-a2d5-e5b277194c2a-kube-api-access-5hrx7\") pod \"payload-pre-processing-e2e-inf-b-bb6e74-6dfbdcd5d-bmst9\" (UID: \"e38f1710-8367-4e0d-a2d5-e5b277194c2a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bb6e74-6dfbdcd5d-bmst9"
Sep 12 00:46:41.310203 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:41.310185    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e38f1710-8367-4e0d-a2d5-e5b277194c2a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-bb6e74-6dfbdcd5d-bmst9\" (UID: \"e38f1710-8367-4e0d-a2d5-e5b277194c2a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bb6e74-6dfbdcd5d-bmst9"
Sep 12 00:46:41.317098 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:41.317064    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hrx7\" (UniqueName: \"kubernetes.io/projected/e38f1710-8367-4e0d-a2d5-e5b277194c2a-kube-api-access-5hrx7\") pod \"payload-pre-processing-e2e-inf-b-bb6e74-6dfbdcd5d-bmst9\" (UID: \"e38f1710-8367-4e0d-a2d5-e5b277194c2a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bb6e74-6dfbdcd5d-bmst9"
Sep 12 00:46:41.375555 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:41.375526    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:41.410550 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:41.410512    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/975e8170-7e78-40dd-a439-a24bd4a50165-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-bb6e74-59d9668859-c96ng\" (UID: \"975e8170-7e78-40dd-a439-a24bd4a50165\") " pod="openshift-ingress/payload-processing-e2e-inf-b-bb6e74-59d9668859-c96ng"
Sep 12 00:46:41.410550 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:41.410554    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgjwc\" (UniqueName: \"kubernetes.io/projected/975e8170-7e78-40dd-a439-a24bd4a50165-kube-api-access-kgjwc\") pod \"payload-processing-e2e-inf-b-bb6e74-59d9668859-c96ng\" (UID: \"975e8170-7e78-40dd-a439-a24bd4a50165\") " pod="openshift-ingress/payload-processing-e2e-inf-b-bb6e74-59d9668859-c96ng"
Sep 12 00:46:41.411294 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:41.411262    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/975e8170-7e78-40dd-a439-a24bd4a50165-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-bb6e74-59d9668859-c96ng\" (UID: \"975e8170-7e78-40dd-a439-a24bd4a50165\") " pod="openshift-ingress/payload-processing-e2e-inf-b-bb6e74-59d9668859-c96ng"
Sep 12 00:46:41.422792 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:41.422762    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgjwc\" (UniqueName: \"kubernetes.io/projected/975e8170-7e78-40dd-a439-a24bd4a50165-kube-api-access-kgjwc\") pod \"payload-processing-e2e-inf-b-bb6e74-59d9668859-c96ng\" (UID: \"975e8170-7e78-40dd-a439-a24bd4a50165\") " pod="openshift-ingress/payload-processing-e2e-inf-b-bb6e74-59d9668859-c96ng"
Sep 12 00:46:41.452993 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:41.452959    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bb6e74-6dfbdcd5d-bmst9"
Sep 12 00:46:41.479160 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:41.479128    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:41.517518 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:41.517381    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-bb6e74-59d9668859-c96ng"
Sep 12 00:46:41.728925 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:41.728899    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-bb6e74-6dfbdcd5d-bmst9"]
Sep 12 00:46:41.730513 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:46:41.730487    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode38f1710_8367_4e0d_a2d5_e5b277194c2a.slice/crio-69d98678484002ad75d2963e88da30bb839224b3557c3864fa38f9c3d6d27243 WatchSource:0}: Error finding container 69d98678484002ad75d2963e88da30bb839224b3557c3864fa38f9c3d6d27243: Status 404 returned error can't find the container with id 69d98678484002ad75d2963e88da30bb839224b3557c3864fa38f9c3d6d27243
Sep 12 00:46:41.806496 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:41.806474    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-bb6e74-59d9668859-c96ng"]
Sep 12 00:46:41.808557 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:46:41.808531    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod975e8170_7e78_40dd_a439_a24bd4a50165.slice/crio-de6456555c7bc6e67182e681ad6ca5ead22ce91706f6aba022174fb05eb36806 WatchSource:0}: Error finding container de6456555c7bc6e67182e681ad6ca5ead22ce91706f6aba022174fb05eb36806: Status 404 returned error can't find the container with id de6456555c7bc6e67182e681ad6ca5ead22ce91706f6aba022174fb05eb36806
Sep 12 00:46:42.037722 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:42.037689    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-bb6e74-59d9668859-c96ng" event={"ID":"975e8170-7e78-40dd-a439-a24bd4a50165","Type":"ContainerStarted","Data":"c9b2ef00ddcf4b89f3071df5ab15fcfe219449131a929e79a902e6ac5ab2b4db"}
Sep 12 00:46:42.037917 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:42.037735    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-bb6e74-59d9668859-c96ng" event={"ID":"975e8170-7e78-40dd-a439-a24bd4a50165","Type":"ContainerStarted","Data":"de6456555c7bc6e67182e681ad6ca5ead22ce91706f6aba022174fb05eb36806"}
Sep 12 00:46:42.037917 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:42.037794    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-bb6e74-59d9668859-c96ng"
Sep 12 00:46:42.039337 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:42.039317    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bb6e74-6dfbdcd5d-bmst9" event={"ID":"e38f1710-8367-4e0d-a2d5-e5b277194c2a","Type":"ContainerStarted","Data":"226007b6af187512bfbaf9f87fbd1f2d3b665a47cf6bf6e8cc14bc7b9d8897ec"}
Sep 12 00:46:42.039461 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:42.039347    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bb6e74-6dfbdcd5d-bmst9" event={"ID":"e38f1710-8367-4e0d-a2d5-e5b277194c2a","Type":"ContainerStarted","Data":"69d98678484002ad75d2963e88da30bb839224b3557c3864fa38f9c3d6d27243"}
Sep 12 00:46:42.039461 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:42.039439    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bb6e74-6dfbdcd5d-bmst9"
Sep 12 00:46:42.053216 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:42.053172    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-bb6e74-59d9668859-c96ng" podStartSLOduration=1.053158514 podStartE2EDuration="1.053158514s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:46:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:46:42.052325568 +0000 UTC m=+1038.726384428" watchObservedRunningTime="2026-09-12 00:46:42.053158514 +0000 UTC m=+1038.727217372"
Sep 12 00:46:42.070094 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:42.070021    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bb6e74-6dfbdcd5d-bmst9" podStartSLOduration=1.069998614 podStartE2EDuration="1.069998614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:46:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:46:42.066755981 +0000 UTC m=+1038.740814841" watchObservedRunningTime="2026-09-12 00:46:42.069998614 +0000 UTC m=+1038.744057477"
Sep 12 00:46:42.379036 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:42.379005    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:43.162611 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:43.162579    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:43.867596 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:43.867562    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:44.542571 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:44.542542    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:45.197931 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:45.197891    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:45.979562 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:45.979530    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:46.538917 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:46.538881    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:46.570512 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:46.570455    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-220bac51-d9bccdcb7-ln7qk"]
Sep 12 00:46:46.572649 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:46.572612    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-220bac51-d9bccdcb7-ln7qk" podUID="9501e6dd-0aaa-494b-b7e5-69c5fb24bc64" containerName="payload-processing" containerID="cri-o://466fb32aebece685effe37697fc107345e130837b3bae367c697ad23f1813e61" gracePeriod=30
Sep 12 00:46:46.579161 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:46.579134    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-220bac51-7878986646-zjp7r"]
Sep 12 00:46:46.579409 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:46.579379    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-220bac51-7878986646-zjp7r" podUID="c437a648-3aa7-464c-bacc-e12fa50f3676" containerName="payload-pre-processing" containerID="cri-o://7c8ce5a19c61b76303ecf94da1bf2c2cc1482f7ae5a853de5274b242fae14d5e" gracePeriod=30
Sep 12 00:46:46.598194 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:46.598158    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:46.657620 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:46.657582    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:46.731941 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:46.731896    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:46.789409 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:46.789336    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:46.883665 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:46.883573    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:46.891268 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:46.891067    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-220bac51-d9bccdcb7-ln7qk"
Sep 12 00:46:46.893163 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:46.893135    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-220bac51-7878986646-zjp7r"
Sep 12 00:46:46.971192 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:46.971158    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c437a648-3aa7-464c-bacc-e12fa50f3676-plugins-config-volume\") pod \"c437a648-3aa7-464c-bacc-e12fa50f3676\" (UID: \"c437a648-3aa7-464c-bacc-e12fa50f3676\") "
Sep 12 00:46:46.971367 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:46.971235    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9501e6dd-0aaa-494b-b7e5-69c5fb24bc64-plugins-config-volume\") pod \"9501e6dd-0aaa-494b-b7e5-69c5fb24bc64\" (UID: \"9501e6dd-0aaa-494b-b7e5-69c5fb24bc64\") "
Sep 12 00:46:46.971367 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:46.971254    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6jkl\" (UniqueName: \"kubernetes.io/projected/9501e6dd-0aaa-494b-b7e5-69c5fb24bc64-kube-api-access-b6jkl\") pod \"9501e6dd-0aaa-494b-b7e5-69c5fb24bc64\" (UID: \"9501e6dd-0aaa-494b-b7e5-69c5fb24bc64\") "
Sep 12 00:46:46.971367 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:46.971301    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxxrh\" (UniqueName: \"kubernetes.io/projected/c437a648-3aa7-464c-bacc-e12fa50f3676-kube-api-access-vxxrh\") pod \"c437a648-3aa7-464c-bacc-e12fa50f3676\" (UID: \"c437a648-3aa7-464c-bacc-e12fa50f3676\") "
Sep 12 00:46:46.971553 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:46.971528    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c437a648-3aa7-464c-bacc-e12fa50f3676-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c437a648-3aa7-464c-bacc-e12fa50f3676" (UID: "c437a648-3aa7-464c-bacc-e12fa50f3676"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:46:46.971654 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:46.971629    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9501e6dd-0aaa-494b-b7e5-69c5fb24bc64-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9501e6dd-0aaa-494b-b7e5-69c5fb24bc64" (UID: "9501e6dd-0aaa-494b-b7e5-69c5fb24bc64"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:46:46.973614 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:46.973591    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c437a648-3aa7-464c-bacc-e12fa50f3676-kube-api-access-vxxrh" (OuterVolumeSpecName: "kube-api-access-vxxrh") pod "c437a648-3aa7-464c-bacc-e12fa50f3676" (UID: "c437a648-3aa7-464c-bacc-e12fa50f3676"). InnerVolumeSpecName "kube-api-access-vxxrh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:46:46.973614 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:46.973608    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9501e6dd-0aaa-494b-b7e5-69c5fb24bc64-kube-api-access-b6jkl" (OuterVolumeSpecName: "kube-api-access-b6jkl") pod "9501e6dd-0aaa-494b-b7e5-69c5fb24bc64" (UID: "9501e6dd-0aaa-494b-b7e5-69c5fb24bc64"). InnerVolumeSpecName "kube-api-access-b6jkl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:46:47.067445 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:47.067363    2558 generic.go:356] "Generic (PLEG): container finished" podID="c437a648-3aa7-464c-bacc-e12fa50f3676" containerID="7c8ce5a19c61b76303ecf94da1bf2c2cc1482f7ae5a853de5274b242fae14d5e" exitCode=0
Sep 12 00:46:47.067445 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:47.067404    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-220bac51-7878986646-zjp7r"
Sep 12 00:46:47.067678 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:47.067450    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-220bac51-7878986646-zjp7r" event={"ID":"c437a648-3aa7-464c-bacc-e12fa50f3676","Type":"ContainerDied","Data":"7c8ce5a19c61b76303ecf94da1bf2c2cc1482f7ae5a853de5274b242fae14d5e"}
Sep 12 00:46:47.067678 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:47.067499    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-220bac51-7878986646-zjp7r" event={"ID":"c437a648-3aa7-464c-bacc-e12fa50f3676","Type":"ContainerDied","Data":"a8060e78db6a590fb7ab1e896b956022aa5a0581212e3706a2b13335a8f9b511"}
Sep 12 00:46:47.067678 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:47.067538    2558 scope.go:117] "RemoveContainer" containerID="7c8ce5a19c61b76303ecf94da1bf2c2cc1482f7ae5a853de5274b242fae14d5e"
Sep 12 00:46:47.068984 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:47.068963    2558 generic.go:356] "Generic (PLEG): container finished" podID="9501e6dd-0aaa-494b-b7e5-69c5fb24bc64" containerID="466fb32aebece685effe37697fc107345e130837b3bae367c697ad23f1813e61" exitCode=0
Sep 12 00:46:47.069128 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:47.068994    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-220bac51-d9bccdcb7-ln7qk" event={"ID":"9501e6dd-0aaa-494b-b7e5-69c5fb24bc64","Type":"ContainerDied","Data":"466fb32aebece685effe37697fc107345e130837b3bae367c697ad23f1813e61"}
Sep 12 00:46:47.069128 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:47.069015    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-220bac51-d9bccdcb7-ln7qk"
Sep 12 00:46:47.069128 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:47.069041    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-220bac51-d9bccdcb7-ln7qk" event={"ID":"9501e6dd-0aaa-494b-b7e5-69c5fb24bc64","Type":"ContainerDied","Data":"f7a5ef49c74e68ef9cf4b7c90ea551897da504a9b79c1c1438341a7b826a2567"}
Sep 12 00:46:47.071928 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:47.071909    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9501e6dd-0aaa-494b-b7e5-69c5fb24bc64-plugins-config-volume\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:46:47.071928 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:47.071929    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6jkl\" (UniqueName: \"kubernetes.io/projected/9501e6dd-0aaa-494b-b7e5-69c5fb24bc64-kube-api-access-b6jkl\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:46:47.072049 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:47.071938    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxxrh\" (UniqueName: \"kubernetes.io/projected/c437a648-3aa7-464c-bacc-e12fa50f3676-kube-api-access-vxxrh\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:46:47.072049 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:47.071946    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c437a648-3aa7-464c-bacc-e12fa50f3676-plugins-config-volume\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:46:47.079206 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:47.079079    2558 scope.go:117] "RemoveContainer" containerID="7c8ce5a19c61b76303ecf94da1bf2c2cc1482f7ae5a853de5274b242fae14d5e"
Sep 12 00:46:47.079524 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:46:47.079494    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c8ce5a19c61b76303ecf94da1bf2c2cc1482f7ae5a853de5274b242fae14d5e\": container with ID starting with 7c8ce5a19c61b76303ecf94da1bf2c2cc1482f7ae5a853de5274b242fae14d5e not found: ID does not exist" containerID="7c8ce5a19c61b76303ecf94da1bf2c2cc1482f7ae5a853de5274b242fae14d5e"
Sep 12 00:46:47.079603 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:47.079535    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c8ce5a19c61b76303ecf94da1bf2c2cc1482f7ae5a853de5274b242fae14d5e"} err="failed to get container status \"7c8ce5a19c61b76303ecf94da1bf2c2cc1482f7ae5a853de5274b242fae14d5e\": rpc error: code = NotFound desc = could not find container \"7c8ce5a19c61b76303ecf94da1bf2c2cc1482f7ae5a853de5274b242fae14d5e\": container with ID starting with 7c8ce5a19c61b76303ecf94da1bf2c2cc1482f7ae5a853de5274b242fae14d5e not found: ID does not exist"
Sep 12 00:46:47.079603 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:47.079558    2558 scope.go:117] "RemoveContainer" containerID="466fb32aebece685effe37697fc107345e130837b3bae367c697ad23f1813e61"
Sep 12 00:46:47.087831 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:47.087816    2558 scope.go:117] "RemoveContainer" containerID="466fb32aebece685effe37697fc107345e130837b3bae367c697ad23f1813e61"
Sep 12 00:46:47.088180 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:46:47.088154    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"466fb32aebece685effe37697fc107345e130837b3bae367c697ad23f1813e61\": container with ID starting with 466fb32aebece685effe37697fc107345e130837b3bae367c697ad23f1813e61 not found: ID does not exist" containerID="466fb32aebece685effe37697fc107345e130837b3bae367c697ad23f1813e61"
Sep 12 00:46:47.088239 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:47.088190    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"466fb32aebece685effe37697fc107345e130837b3bae367c697ad23f1813e61"} err="failed to get container status \"466fb32aebece685effe37697fc107345e130837b3bae367c697ad23f1813e61\": rpc error: code = NotFound desc = could not find container \"466fb32aebece685effe37697fc107345e130837b3bae367c697ad23f1813e61\": container with ID starting with 466fb32aebece685effe37697fc107345e130837b3bae367c697ad23f1813e61 not found: ID does not exist"
Sep 12 00:46:47.099744 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:47.099717    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-220bac51-7878986646-zjp7r"]
Sep 12 00:46:47.101962 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:47.101937    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-220bac51-7878986646-zjp7r"]
Sep 12 00:46:47.127770 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:47.127736    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-220bac51-d9bccdcb7-ln7qk"]
Sep 12 00:46:47.138256 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:47.137756    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-220bac51-d9bccdcb7-ln7qk"]
Sep 12 00:46:47.236353 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:47.236325    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:47.355597 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:47.355519    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:47.452727 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:47.452684    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:47.558509 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:47.558477    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:47.738933 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:47.738902    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9501e6dd-0aaa-494b-b7e5-69c5fb24bc64" path="/var/lib/kubelet/pods/9501e6dd-0aaa-494b-b7e5-69c5fb24bc64/volumes"
Sep 12 00:46:47.739281 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:47.739267    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c437a648-3aa7-464c-bacc-e12fa50f3676" path="/var/lib/kubelet/pods/c437a648-3aa7-464c-bacc-e12fa50f3676/volumes"
Sep 12 00:46:47.864346 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:47.864309    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:48.070210 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:48.070116    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:48.175229 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:48.175195    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:48.868851 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:48.868823    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:49.673028 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:49.672987    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:50.217356 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:50.217319    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:50.315048 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:50.315011    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:50.415854 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:50.415822    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:51.148281 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:51.148248    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:52.068281 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:52.068249    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:52.141526 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:52.141489    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:52.290205 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:52.290171    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:52.640927 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:52.640899    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:52.737441 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:52.737408    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:52.743348 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:52.743319    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-723148b6-9d745c6bb-pkf2x"]
Sep 12 00:46:52.743910 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:52.743892    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c437a648-3aa7-464c-bacc-e12fa50f3676" containerName="payload-pre-processing"
Sep 12 00:46:52.743998 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:52.743914    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c437a648-3aa7-464c-bacc-e12fa50f3676" containerName="payload-pre-processing"
Sep 12 00:46:52.743998 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:52.743931    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9501e6dd-0aaa-494b-b7e5-69c5fb24bc64" containerName="payload-processing"
Sep 12 00:46:52.743998 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:52.743938    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9501e6dd-0aaa-494b-b7e5-69c5fb24bc64" containerName="payload-processing"
Sep 12 00:46:52.744118 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:52.744012    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="c437a648-3aa7-464c-bacc-e12fa50f3676" containerName="payload-pre-processing"
Sep 12 00:46:52.744118 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:52.744022    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9501e6dd-0aaa-494b-b7e5-69c5fb24bc64" containerName="payload-processing"
Sep 12 00:46:52.747517 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:52.747492    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-723148b6-9d745c6bb-pkf2x"
Sep 12 00:46:52.750067 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:52.750047    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-723148b6-dockercfg-s2d7b\""
Sep 12 00:46:52.750174 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:52.750049    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-723148b6\""
Sep 12 00:46:52.762621 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:52.762585    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-723148b6-9d745c6bb-pkf2x"]
Sep 12 00:46:52.781831 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:52.781799    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-723148b6-7d96755558-tbp24"]
Sep 12 00:46:52.785595 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:52.785572    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-723148b6-7d96755558-tbp24"
Sep 12 00:46:52.795348 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:52.795314    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-723148b6-7d96755558-tbp24"]
Sep 12 00:46:52.833877 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:52.833827    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhtdb\" (UniqueName: \"kubernetes.io/projected/164869dc-3b3d-4d2d-8f3a-ad0e652df02a-kube-api-access-qhtdb\") pod \"payload-pre-processing-e2e-migrate-723148b6-9d745c6bb-pkf2x\" (UID: \"164869dc-3b3d-4d2d-8f3a-ad0e652df02a\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-723148b6-9d745c6bb-pkf2x"
Sep 12 00:46:52.834321 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:52.834286    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbgwl\" (UniqueName: \"kubernetes.io/projected/415e62b4-3c7f-4039-b613-bfce166bdd35-kube-api-access-pbgwl\") pod \"payload-processing-e2e-migrate-723148b6-7d96755558-tbp24\" (UID: \"415e62b4-3c7f-4039-b613-bfce166bdd35\") " pod="openshift-ingress/payload-processing-e2e-migrate-723148b6-7d96755558-tbp24"
Sep 12 00:46:52.834563 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:52.834533    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/415e62b4-3c7f-4039-b613-bfce166bdd35-plugins-config-volume\") pod \"payload-processing-e2e-migrate-723148b6-7d96755558-tbp24\" (UID: \"415e62b4-3c7f-4039-b613-bfce166bdd35\") " pod="openshift-ingress/payload-processing-e2e-migrate-723148b6-7d96755558-tbp24"
Sep 12 00:46:52.834680 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:52.834615    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/164869dc-3b3d-4d2d-8f3a-ad0e652df02a-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-723148b6-9d745c6bb-pkf2x\" (UID: \"164869dc-3b3d-4d2d-8f3a-ad0e652df02a\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-723148b6-9d745c6bb-pkf2x"
Sep 12 00:46:52.935492 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:52.935461    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhtdb\" (UniqueName: \"kubernetes.io/projected/164869dc-3b3d-4d2d-8f3a-ad0e652df02a-kube-api-access-qhtdb\") pod \"payload-pre-processing-e2e-migrate-723148b6-9d745c6bb-pkf2x\" (UID: \"164869dc-3b3d-4d2d-8f3a-ad0e652df02a\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-723148b6-9d745c6bb-pkf2x"
Sep 12 00:46:52.935681 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:52.935526    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbgwl\" (UniqueName: \"kubernetes.io/projected/415e62b4-3c7f-4039-b613-bfce166bdd35-kube-api-access-pbgwl\") pod \"payload-processing-e2e-migrate-723148b6-7d96755558-tbp24\" (UID: \"415e62b4-3c7f-4039-b613-bfce166bdd35\") " pod="openshift-ingress/payload-processing-e2e-migrate-723148b6-7d96755558-tbp24"
Sep 12 00:46:52.935681 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:52.935555    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/415e62b4-3c7f-4039-b613-bfce166bdd35-plugins-config-volume\") pod \"payload-processing-e2e-migrate-723148b6-7d96755558-tbp24\" (UID: \"415e62b4-3c7f-4039-b613-bfce166bdd35\") " pod="openshift-ingress/payload-processing-e2e-migrate-723148b6-7d96755558-tbp24"
Sep 12 00:46:52.935681 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:52.935587    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/164869dc-3b3d-4d2d-8f3a-ad0e652df02a-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-723148b6-9d745c6bb-pkf2x\" (UID: \"164869dc-3b3d-4d2d-8f3a-ad0e652df02a\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-723148b6-9d745c6bb-pkf2x"
Sep 12 00:46:52.936356 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:52.936334    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/164869dc-3b3d-4d2d-8f3a-ad0e652df02a-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-723148b6-9d745c6bb-pkf2x\" (UID: \"164869dc-3b3d-4d2d-8f3a-ad0e652df02a\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-723148b6-9d745c6bb-pkf2x"
Sep 12 00:46:52.936536 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:52.936513    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/415e62b4-3c7f-4039-b613-bfce166bdd35-plugins-config-volume\") pod \"payload-processing-e2e-migrate-723148b6-7d96755558-tbp24\" (UID: \"415e62b4-3c7f-4039-b613-bfce166bdd35\") " pod="openshift-ingress/payload-processing-e2e-migrate-723148b6-7d96755558-tbp24"
Sep 12 00:46:52.943648 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:52.943612    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbgwl\" (UniqueName: \"kubernetes.io/projected/415e62b4-3c7f-4039-b613-bfce166bdd35-kube-api-access-pbgwl\") pod \"payload-processing-e2e-migrate-723148b6-7d96755558-tbp24\" (UID: \"415e62b4-3c7f-4039-b613-bfce166bdd35\") " pod="openshift-ingress/payload-processing-e2e-migrate-723148b6-7d96755558-tbp24"
Sep 12 00:46:52.943759 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:52.943612    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhtdb\" (UniqueName: \"kubernetes.io/projected/164869dc-3b3d-4d2d-8f3a-ad0e652df02a-kube-api-access-qhtdb\") pod \"payload-pre-processing-e2e-migrate-723148b6-9d745c6bb-pkf2x\" (UID: \"164869dc-3b3d-4d2d-8f3a-ad0e652df02a\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-723148b6-9d745c6bb-pkf2x"
Sep 12 00:46:53.046528 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:53.046497    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-bb6e74-59d9668859-c96ng"
Sep 12 00:46:53.046738 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:53.046636    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bb6e74-6dfbdcd5d-bmst9"
Sep 12 00:46:53.057992 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:53.057965    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-723148b6-9d745c6bb-pkf2x"
Sep 12 00:46:53.074958 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:53.074913    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:53.096897 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:53.096864    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-723148b6-7d96755558-tbp24"
Sep 12 00:46:53.345738 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:53.345716    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-723148b6-9d745c6bb-pkf2x"]
Sep 12 00:46:53.347720 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:46:53.347697    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod164869dc_3b3d_4d2d_8f3a_ad0e652df02a.slice/crio-ac80bb55170abd530eeb032b048e8753d8debd0dac70f31d16d570d6e9e48fc8 WatchSource:0}: Error finding container ac80bb55170abd530eeb032b048e8753d8debd0dac70f31d16d570d6e9e48fc8: Status 404 returned error can't find the container with id ac80bb55170abd530eeb032b048e8753d8debd0dac70f31d16d570d6e9e48fc8
Sep 12 00:46:53.386953 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:53.386929    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-723148b6-7d96755558-tbp24"]
Sep 12 00:46:53.391822 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:46:53.391794    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod415e62b4_3c7f_4039_b613_bfce166bdd35.slice/crio-164f47bd99e3a1b1710eaed9f08da4aea91caf98226202c3a15b6a6d1ff642d1 WatchSource:0}: Error finding container 164f47bd99e3a1b1710eaed9f08da4aea91caf98226202c3a15b6a6d1ff642d1: Status 404 returned error can't find the container with id 164f47bd99e3a1b1710eaed9f08da4aea91caf98226202c3a15b6a6d1ff642d1
Sep 12 00:46:54.001412 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:54.001376    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:54.115928 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:54.115881    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-723148b6-7d96755558-tbp24" event={"ID":"415e62b4-3c7f-4039-b613-bfce166bdd35","Type":"ContainerStarted","Data":"6bdeaecfe99cff7c3d682e1ac48977dbd57c5e5ac9e6f5d0257aafd4563fe8d1"}
Sep 12 00:46:54.116447 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:54.116079    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-723148b6-7d96755558-tbp24"
Sep 12 00:46:54.116447 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:54.116139    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-723148b6-7d96755558-tbp24" event={"ID":"415e62b4-3c7f-4039-b613-bfce166bdd35","Type":"ContainerStarted","Data":"164f47bd99e3a1b1710eaed9f08da4aea91caf98226202c3a15b6a6d1ff642d1"}
Sep 12 00:46:54.117643 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:54.117623    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-723148b6-9d745c6bb-pkf2x" event={"ID":"164869dc-3b3d-4d2d-8f3a-ad0e652df02a","Type":"ContainerStarted","Data":"76f0652255bf2423d634c5faa4c4b1a998d8e4c9ff3033d542d1c98a8319dfa0"}
Sep 12 00:46:54.117734 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:54.117658    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-723148b6-9d745c6bb-pkf2x" event={"ID":"164869dc-3b3d-4d2d-8f3a-ad0e652df02a","Type":"ContainerStarted","Data":"ac80bb55170abd530eeb032b048e8753d8debd0dac70f31d16d570d6e9e48fc8"}
Sep 12 00:46:54.117772 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:54.117756    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-723148b6-9d745c6bb-pkf2x"
Sep 12 00:46:54.134725 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:54.134667    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-723148b6-7d96755558-tbp24" podStartSLOduration=2.134646748 podStartE2EDuration="2.134646748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:46:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:46:54.131437228 +0000 UTC m=+1050.805496089" watchObservedRunningTime="2026-09-12 00:46:54.134646748 +0000 UTC m=+1050.808705612"
Sep 12 00:46:54.149634 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:54.149578    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-723148b6-9d745c6bb-pkf2x" podStartSLOduration=2.149562006 podStartE2EDuration="2.149562006s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:46:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:46:54.147673217 +0000 UTC m=+1050.821732077" watchObservedRunningTime="2026-09-12 00:46:54.149562006 +0000 UTC m=+1050.823620864"
Sep 12 00:46:54.741343 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:54.741306    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:55.555393 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:55.555361    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:55.629155 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:55.629119    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:55.998442 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:55.998413    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:56.253542 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:56.253462    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:56.914883 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:56.914851    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:57.243485 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:57.243453    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:46:57.677778 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:46:57.677676    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:01.419878 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:01.419837    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:05.125541 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:05.125509    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-723148b6-9d745c6bb-pkf2x"
Sep 12 00:47:05.125996 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:05.125582    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-723148b6-7d96755558-tbp24"
Sep 12 00:47:13.143655 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:13.143617    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:14.246874 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:14.246837    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:14.873235 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:14.873200    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:15.696597 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:15.696493    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:16.482070 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:16.482037    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:17.101800 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:17.101761    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:17.756506 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:17.756472    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:18.475142 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:18.475111    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:19.087043 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:19.086994    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:19.967582 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:19.967548    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:21.340864 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:21.340824    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:21.889419 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:21.889391    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:21.919882 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:21.919838    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-723148b6-7d96755558-tbp24"]
Sep 12 00:47:21.920161 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:21.920130    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-723148b6-7d96755558-tbp24" podUID="415e62b4-3c7f-4039-b613-bfce166bdd35" containerName="payload-processing" containerID="cri-o://6bdeaecfe99cff7c3d682e1ac48977dbd57c5e5ac9e6f5d0257aafd4563fe8d1" gracePeriod=30
Sep 12 00:47:21.924920 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:21.924869    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-723148b6-9d745c6bb-pkf2x"]
Sep 12 00:47:21.925374 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:21.925345    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-723148b6-9d745c6bb-pkf2x" podUID="164869dc-3b3d-4d2d-8f3a-ad0e652df02a" containerName="payload-pre-processing" containerID="cri-o://76f0652255bf2423d634c5faa4c4b1a998d8e4c9ff3033d542d1c98a8319dfa0" gracePeriod=30
Sep 12 00:47:21.989569 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:21.989534    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:22.056190 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.056160    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:22.113664 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.113629    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:22.205905 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.204185    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:22.215112 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.215047    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-723148b6-7d96755558-tbp24"
Sep 12 00:47:22.246412 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.246373    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-723148b6-9d745c6bb-pkf2x"
Sep 12 00:47:22.273364 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.273327    2558 generic.go:356] "Generic (PLEG): container finished" podID="164869dc-3b3d-4d2d-8f3a-ad0e652df02a" containerID="76f0652255bf2423d634c5faa4c4b1a998d8e4c9ff3033d542d1c98a8319dfa0" exitCode=0
Sep 12 00:47:22.273547 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.273388    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-723148b6-9d745c6bb-pkf2x"
Sep 12 00:47:22.273547 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.273406    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-723148b6-9d745c6bb-pkf2x" event={"ID":"164869dc-3b3d-4d2d-8f3a-ad0e652df02a","Type":"ContainerDied","Data":"76f0652255bf2423d634c5faa4c4b1a998d8e4c9ff3033d542d1c98a8319dfa0"}
Sep 12 00:47:22.273547 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.273489    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-723148b6-9d745c6bb-pkf2x" event={"ID":"164869dc-3b3d-4d2d-8f3a-ad0e652df02a","Type":"ContainerDied","Data":"ac80bb55170abd530eeb032b048e8753d8debd0dac70f31d16d570d6e9e48fc8"}
Sep 12 00:47:22.273547 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.273524    2558 scope.go:117] "RemoveContainer" containerID="76f0652255bf2423d634c5faa4c4b1a998d8e4c9ff3033d542d1c98a8319dfa0"
Sep 12 00:47:22.275180 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.275154    2558 generic.go:356] "Generic (PLEG): container finished" podID="415e62b4-3c7f-4039-b613-bfce166bdd35" containerID="6bdeaecfe99cff7c3d682e1ac48977dbd57c5e5ac9e6f5d0257aafd4563fe8d1" exitCode=0
Sep 12 00:47:22.275463 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.275238    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-723148b6-7d96755558-tbp24"
Sep 12 00:47:22.275463 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.275260    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-723148b6-7d96755558-tbp24" event={"ID":"415e62b4-3c7f-4039-b613-bfce166bdd35","Type":"ContainerDied","Data":"6bdeaecfe99cff7c3d682e1ac48977dbd57c5e5ac9e6f5d0257aafd4563fe8d1"}
Sep 12 00:47:22.275463 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.275301    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-723148b6-7d96755558-tbp24" event={"ID":"415e62b4-3c7f-4039-b613-bfce166bdd35","Type":"ContainerDied","Data":"164f47bd99e3a1b1710eaed9f08da4aea91caf98226202c3a15b6a6d1ff642d1"}
Sep 12 00:47:22.282911 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.282885    2558 scope.go:117] "RemoveContainer" containerID="76f0652255bf2423d634c5faa4c4b1a998d8e4c9ff3033d542d1c98a8319dfa0"
Sep 12 00:47:22.283230 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:47:22.283211    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76f0652255bf2423d634c5faa4c4b1a998d8e4c9ff3033d542d1c98a8319dfa0\": container with ID starting with 76f0652255bf2423d634c5faa4c4b1a998d8e4c9ff3033d542d1c98a8319dfa0 not found: ID does not exist" containerID="76f0652255bf2423d634c5faa4c4b1a998d8e4c9ff3033d542d1c98a8319dfa0"
Sep 12 00:47:22.283317 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.283241    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76f0652255bf2423d634c5faa4c4b1a998d8e4c9ff3033d542d1c98a8319dfa0"} err="failed to get container status \"76f0652255bf2423d634c5faa4c4b1a998d8e4c9ff3033d542d1c98a8319dfa0\": rpc error: code = NotFound desc = could not find container \"76f0652255bf2423d634c5faa4c4b1a998d8e4c9ff3033d542d1c98a8319dfa0\": container with ID starting with 76f0652255bf2423d634c5faa4c4b1a998d8e4c9ff3033d542d1c98a8319dfa0 not found: ID does not exist"
Sep 12 00:47:22.283317 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.283263    2558 scope.go:117] "RemoveContainer" containerID="6bdeaecfe99cff7c3d682e1ac48977dbd57c5e5ac9e6f5d0257aafd4563fe8d1"
Sep 12 00:47:22.293580 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.293555    2558 scope.go:117] "RemoveContainer" containerID="6bdeaecfe99cff7c3d682e1ac48977dbd57c5e5ac9e6f5d0257aafd4563fe8d1"
Sep 12 00:47:22.293920 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:47:22.293894    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bdeaecfe99cff7c3d682e1ac48977dbd57c5e5ac9e6f5d0257aafd4563fe8d1\": container with ID starting with 6bdeaecfe99cff7c3d682e1ac48977dbd57c5e5ac9e6f5d0257aafd4563fe8d1 not found: ID does not exist" containerID="6bdeaecfe99cff7c3d682e1ac48977dbd57c5e5ac9e6f5d0257aafd4563fe8d1"
Sep 12 00:47:22.294018 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.293925    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bdeaecfe99cff7c3d682e1ac48977dbd57c5e5ac9e6f5d0257aafd4563fe8d1"} err="failed to get container status \"6bdeaecfe99cff7c3d682e1ac48977dbd57c5e5ac9e6f5d0257aafd4563fe8d1\": rpc error: code = NotFound desc = could not find container \"6bdeaecfe99cff7c3d682e1ac48977dbd57c5e5ac9e6f5d0257aafd4563fe8d1\": container with ID starting with 6bdeaecfe99cff7c3d682e1ac48977dbd57c5e5ac9e6f5d0257aafd4563fe8d1 not found: ID does not exist"
Sep 12 00:47:22.346608 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.346572    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pbgwl\" (UniqueName: \"kubernetes.io/projected/415e62b4-3c7f-4039-b613-bfce166bdd35-kube-api-access-pbgwl\") pod \"415e62b4-3c7f-4039-b613-bfce166bdd35\" (UID: \"415e62b4-3c7f-4039-b613-bfce166bdd35\") "
Sep 12 00:47:22.347068 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.346622    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/164869dc-3b3d-4d2d-8f3a-ad0e652df02a-plugins-config-volume\") pod \"164869dc-3b3d-4d2d-8f3a-ad0e652df02a\" (UID: \"164869dc-3b3d-4d2d-8f3a-ad0e652df02a\") "
Sep 12 00:47:22.347068 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.346668    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/415e62b4-3c7f-4039-b613-bfce166bdd35-plugins-config-volume\") pod \"415e62b4-3c7f-4039-b613-bfce166bdd35\" (UID: \"415e62b4-3c7f-4039-b613-bfce166bdd35\") "
Sep 12 00:47:22.347068 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.346720    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhtdb\" (UniqueName: \"kubernetes.io/projected/164869dc-3b3d-4d2d-8f3a-ad0e652df02a-kube-api-access-qhtdb\") pod \"164869dc-3b3d-4d2d-8f3a-ad0e652df02a\" (UID: \"164869dc-3b3d-4d2d-8f3a-ad0e652df02a\") "
Sep 12 00:47:22.347253 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.347070    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/164869dc-3b3d-4d2d-8f3a-ad0e652df02a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "164869dc-3b3d-4d2d-8f3a-ad0e652df02a" (UID: "164869dc-3b3d-4d2d-8f3a-ad0e652df02a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:47:22.347253 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.347130    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/415e62b4-3c7f-4039-b613-bfce166bdd35-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "415e62b4-3c7f-4039-b613-bfce166bdd35" (UID: "415e62b4-3c7f-4039-b613-bfce166bdd35"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:47:22.348931 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.348911    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/164869dc-3b3d-4d2d-8f3a-ad0e652df02a-kube-api-access-qhtdb" (OuterVolumeSpecName: "kube-api-access-qhtdb") pod "164869dc-3b3d-4d2d-8f3a-ad0e652df02a" (UID: "164869dc-3b3d-4d2d-8f3a-ad0e652df02a"). InnerVolumeSpecName "kube-api-access-qhtdb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:47:22.348991 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.348939    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/415e62b4-3c7f-4039-b613-bfce166bdd35-kube-api-access-pbgwl" (OuterVolumeSpecName: "kube-api-access-pbgwl") pod "415e62b4-3c7f-4039-b613-bfce166bdd35" (UID: "415e62b4-3c7f-4039-b613-bfce166bdd35"). InnerVolumeSpecName "kube-api-access-pbgwl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:47:22.447659 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.447626    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/415e62b4-3c7f-4039-b613-bfce166bdd35-plugins-config-volume\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:47:22.447659 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.447658    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhtdb\" (UniqueName: \"kubernetes.io/projected/164869dc-3b3d-4d2d-8f3a-ad0e652df02a-kube-api-access-qhtdb\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:47:22.447856 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.447669    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbgwl\" (UniqueName: \"kubernetes.io/projected/415e62b4-3c7f-4039-b613-bfce166bdd35-kube-api-access-pbgwl\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:47:22.447856 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.447679    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/164869dc-3b3d-4d2d-8f3a-ad0e652df02a-plugins-config-volume\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:47:22.484366 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.484334    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:22.596944 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.596919    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-723148b6-9d745c6bb-pkf2x"]
Sep 12 00:47:22.600031 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.600008    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-723148b6-9d745c6bb-pkf2x"]
Sep 12 00:47:22.609028 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.609007    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-723148b6-7d96755558-tbp24"]
Sep 12 00:47:22.610955 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.610933    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-723148b6-7d96755558-tbp24"]
Sep 12 00:47:22.649752 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:22.649720    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:23.013882 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:23.013845    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:23.182508 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:23.182448    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:23.350417 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:23.350338    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:23.738948 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:23.738914    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="164869dc-3b3d-4d2d-8f3a-ad0e652df02a" path="/var/lib/kubelet/pods/164869dc-3b3d-4d2d-8f3a-ad0e652df02a/volumes"
Sep 12 00:47:23.739373 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:23.739356    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="415e62b4-3c7f-4039-b613-bfce166bdd35" path="/var/lib/kubelet/pods/415e62b4-3c7f-4039-b613-bfce166bdd35/volumes"
Sep 12 00:47:23.983623 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:23.983582    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:24.701099 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:24.701054    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:25.281384 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:25.281351    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:25.381480 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:25.381450    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:25.482181 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:25.482147    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:26.286793 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:26.286759    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:27.014423 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:27.014388    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:27.114022 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:27.113992    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:27.371544 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:27.371439    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:27.495120 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:27.495060    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:27.591835 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:27.591799    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:28.072596 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:28.072560    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:28.701143 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:28.701105    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:29.454920 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:29.454887    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:30.122463 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:30.122428    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:33.606021 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:33.605992    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:40.418915 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:40.418866    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:48.503463 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:48.503428    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f"]
Sep 12 00:47:48.503973 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:48.503913    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="415e62b4-3c7f-4039-b613-bfce166bdd35" containerName="payload-processing"
Sep 12 00:47:48.503973 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:48.503928    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="415e62b4-3c7f-4039-b613-bfce166bdd35" containerName="payload-processing"
Sep 12 00:47:48.503973 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:48.503956    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="164869dc-3b3d-4d2d-8f3a-ad0e652df02a" containerName="payload-pre-processing"
Sep 12 00:47:48.503973 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:48.503961    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="164869dc-3b3d-4d2d-8f3a-ad0e652df02a" containerName="payload-pre-processing"
Sep 12 00:47:48.504154 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:48.504062    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="164869dc-3b3d-4d2d-8f3a-ad0e652df02a" containerName="payload-pre-processing"
Sep 12 00:47:48.504154 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:48.504072    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="415e62b4-3c7f-4039-b613-bfce166bdd35" containerName="payload-processing"
Sep 12 00:47:48.509012 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:48.508970    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f"
Sep 12 00:47:48.512721 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:48.512692    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-4b6ed7\"/\"default-dockercfg-l5758\""
Sep 12 00:47:48.512721 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:48.512712    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-4b6ed7\"/\"autores-model-4b6ed7-kserve-self-signed-certs\""
Sep 12 00:47:48.512916 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:48.512691    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-4b6ed7\"/\"openshift-service-ca.crt\""
Sep 12 00:47:48.513004 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:48.512986    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-4b6ed7\"/\"kube-root-ca.crt\""
Sep 12 00:47:48.517141 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:48.517118    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f"]
Sep 12 00:47:48.598686 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:48.598653    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e68f3ff0-dc53-4295-b634-44e0a6bb272f-model-cache\") pod \"autores-model-4b6ed7-kserve-6f64967cf9-8n29f\" (UID: \"e68f3ff0-dc53-4295-b634-44e0a6bb272f\") " pod="ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f"
Sep 12 00:47:48.598686 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:48.598694    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e68f3ff0-dc53-4295-b634-44e0a6bb272f-home\") pod \"autores-model-4b6ed7-kserve-6f64967cf9-8n29f\" (UID: \"e68f3ff0-dc53-4295-b634-44e0a6bb272f\") " pod="ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f"
Sep 12 00:47:48.598938 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:48.598717    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e68f3ff0-dc53-4295-b634-44e0a6bb272f-dshm\") pod \"autores-model-4b6ed7-kserve-6f64967cf9-8n29f\" (UID: \"e68f3ff0-dc53-4295-b634-44e0a6bb272f\") " pod="ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f"
Sep 12 00:47:48.598938 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:48.598831    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l92fw\" (UniqueName: \"kubernetes.io/projected/e68f3ff0-dc53-4295-b634-44e0a6bb272f-kube-api-access-l92fw\") pod \"autores-model-4b6ed7-kserve-6f64967cf9-8n29f\" (UID: \"e68f3ff0-dc53-4295-b634-44e0a6bb272f\") " pod="ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f"
Sep 12 00:47:48.598938 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:48.598871    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e68f3ff0-dc53-4295-b634-44e0a6bb272f-tls-certs\") pod \"autores-model-4b6ed7-kserve-6f64967cf9-8n29f\" (UID: \"e68f3ff0-dc53-4295-b634-44e0a6bb272f\") " pod="ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f"
Sep 12 00:47:48.598938 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:48.598889    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e68f3ff0-dc53-4295-b634-44e0a6bb272f-tmp-dir\") pod \"autores-model-4b6ed7-kserve-6f64967cf9-8n29f\" (UID: \"e68f3ff0-dc53-4295-b634-44e0a6bb272f\") " pod="ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f"
Sep 12 00:47:48.699947 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:48.699913    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e68f3ff0-dc53-4295-b634-44e0a6bb272f-home\") pod \"autores-model-4b6ed7-kserve-6f64967cf9-8n29f\" (UID: \"e68f3ff0-dc53-4295-b634-44e0a6bb272f\") " pod="ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f"
Sep 12 00:47:48.699947 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:48.699953    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e68f3ff0-dc53-4295-b634-44e0a6bb272f-dshm\") pod \"autores-model-4b6ed7-kserve-6f64967cf9-8n29f\" (UID: \"e68f3ff0-dc53-4295-b634-44e0a6bb272f\") " pod="ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f"
Sep 12 00:47:48.700185 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:48.700013    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l92fw\" (UniqueName: \"kubernetes.io/projected/e68f3ff0-dc53-4295-b634-44e0a6bb272f-kube-api-access-l92fw\") pod \"autores-model-4b6ed7-kserve-6f64967cf9-8n29f\" (UID: \"e68f3ff0-dc53-4295-b634-44e0a6bb272f\") " pod="ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f"
Sep 12 00:47:48.700185 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:48.700043    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e68f3ff0-dc53-4295-b634-44e0a6bb272f-tls-certs\") pod \"autores-model-4b6ed7-kserve-6f64967cf9-8n29f\" (UID: \"e68f3ff0-dc53-4295-b634-44e0a6bb272f\") " pod="ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f"
Sep 12 00:47:48.700185 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:48.700066    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e68f3ff0-dc53-4295-b634-44e0a6bb272f-tmp-dir\") pod \"autores-model-4b6ed7-kserve-6f64967cf9-8n29f\" (UID: \"e68f3ff0-dc53-4295-b634-44e0a6bb272f\") " pod="ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f"
Sep 12 00:47:48.700185 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:48.700142    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e68f3ff0-dc53-4295-b634-44e0a6bb272f-model-cache\") pod \"autores-model-4b6ed7-kserve-6f64967cf9-8n29f\" (UID: \"e68f3ff0-dc53-4295-b634-44e0a6bb272f\") " pod="ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f"
Sep 12 00:47:48.700392 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:48.700367    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e68f3ff0-dc53-4295-b634-44e0a6bb272f-home\") pod \"autores-model-4b6ed7-kserve-6f64967cf9-8n29f\" (UID: \"e68f3ff0-dc53-4295-b634-44e0a6bb272f\") " pod="ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f"
Sep 12 00:47:48.700482 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:48.700452    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e68f3ff0-dc53-4295-b634-44e0a6bb272f-model-cache\") pod \"autores-model-4b6ed7-kserve-6f64967cf9-8n29f\" (UID: \"e68f3ff0-dc53-4295-b634-44e0a6bb272f\") " pod="ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f"
Sep 12 00:47:48.700482 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:48.700474    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e68f3ff0-dc53-4295-b634-44e0a6bb272f-tmp-dir\") pod \"autores-model-4b6ed7-kserve-6f64967cf9-8n29f\" (UID: \"e68f3ff0-dc53-4295-b634-44e0a6bb272f\") " pod="ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f"
Sep 12 00:47:48.702442 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:48.702423    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e68f3ff0-dc53-4295-b634-44e0a6bb272f-dshm\") pod \"autores-model-4b6ed7-kserve-6f64967cf9-8n29f\" (UID: \"e68f3ff0-dc53-4295-b634-44e0a6bb272f\") " pod="ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f"
Sep 12 00:47:48.702710 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:48.702690    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e68f3ff0-dc53-4295-b634-44e0a6bb272f-tls-certs\") pod \"autores-model-4b6ed7-kserve-6f64967cf9-8n29f\" (UID: \"e68f3ff0-dc53-4295-b634-44e0a6bb272f\") " pod="ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f"
Sep 12 00:47:48.718079 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:48.718047    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l92fw\" (UniqueName: \"kubernetes.io/projected/e68f3ff0-dc53-4295-b634-44e0a6bb272f-kube-api-access-l92fw\") pod \"autores-model-4b6ed7-kserve-6f64967cf9-8n29f\" (UID: \"e68f3ff0-dc53-4295-b634-44e0a6bb272f\") " pod="ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f"
Sep 12 00:47:48.819355 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:48.819272    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f"
Sep 12 00:47:48.824759 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:48.824734    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:49.009311 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:49.009264    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:49.096145 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:49.095948    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f"]
Sep 12 00:47:49.099635 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:47:49.099606    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode68f3ff0_dc53_4295_b634_44e0a6bb272f.slice/crio-f691ca15653e7fe73e43d61b3d0088232d11f0e1ec9e0f3eb7e0e3cb572fc9ca WatchSource:0}: Error finding container f691ca15653e7fe73e43d61b3d0088232d11f0e1ec9e0f3eb7e0e3cb572fc9ca: Status 404 returned error can't find the container with id f691ca15653e7fe73e43d61b3d0088232d11f0e1ec9e0f3eb7e0e3cb572fc9ca
Sep 12 00:47:49.428193 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:49.427272    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f" event={"ID":"e68f3ff0-dc53-4295-b634-44e0a6bb272f","Type":"ContainerStarted","Data":"bab816f15c2c9985eb57547102f12f8bce35f1075e36d7a44e3acb20bc98526f"}
Sep 12 00:47:49.428193 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:49.427332    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f" event={"ID":"e68f3ff0-dc53-4295-b634-44e0a6bb272f","Type":"ContainerStarted","Data":"f691ca15653e7fe73e43d61b3d0088232d11f0e1ec9e0f3eb7e0e3cb572fc9ca"}
Sep 12 00:47:49.445880 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:49.445822    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f" podStartSLOduration=1.445806615 podStartE2EDuration="1.445806615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:47:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:47:49.443352256 +0000 UTC m=+1106.117411116" watchObservedRunningTime="2026-09-12 00:47:49.445806615 +0000 UTC m=+1106.119865474"
Sep 12 00:47:49.766195 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:49.766163    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:50.601448 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:50.601421    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:51.545652 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:51.545610    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:52.235872 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:52.235814    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:52.966238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:52.966202    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:53.699585 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:53.699549    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:54.368342 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:54.368299    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:55.087647 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:55.087610    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:55.775482 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:55.775452    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:56.581913 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:56.581884    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:57.139300 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:57.139269    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:57.239530 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:57.239487    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:57.309633 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:57.309597    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:57.361163 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:57.361129    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:57.740153 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:57.740120    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:58.409921 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:58.409874    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:58.504961 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:58.504928    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:58.671952 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:58.671865    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:58.772755 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:58.772720    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:58.819984 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:58.819954    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f"
Sep 12 00:47:58.820192 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:58.820007    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f"
Sep 12 00:47:58.833289 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:58.833266    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f"
Sep 12 00:47:58.852604 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:58.852565    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:59.085061 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:59.085012    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:59.182400 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:59.182370    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:59.318072 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:59.318043    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:47:59.490722 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:59.490687    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f"
Sep 12 00:47:59.810677 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:47:59.810596    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:00.765275 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:00.765228    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:01.453525 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:01.453483    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:01.582001 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:01.581959    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:01.679156 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:01.679127    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:02.579520 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:02.579481    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:03.336683 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:03.336648    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:04.143345 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:04.143312    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:04.790664 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:04.790630    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:05.443150 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:05.443115    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:06.192267 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:06.192217    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:06.292966 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:06.292934    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:06.400155 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:06.400127    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:06.744678 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:06.744650    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:06.824829 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:06.824792    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:10.094825 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:10.094794    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5"]
Sep 12 00:48:10.098572 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:10.098555    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5"
Sep 12 00:48:10.100987 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:10.100967    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-81df64\"/\"openshift-service-ca.crt\""
Sep 12 00:48:10.102007 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:10.101990    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-81df64\"/\"test-model-81df64-kserve-self-signed-certs\""
Sep 12 00:48:10.102158 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:10.102143    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-81df64\"/\"default-dockercfg-sf7gj\""
Sep 12 00:48:10.102237 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:10.102220    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-81df64\"/\"kube-root-ca.crt\""
Sep 12 00:48:10.107932 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:10.107908    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5"]
Sep 12 00:48:10.220138 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:10.220064    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4md7\" (UniqueName: \"kubernetes.io/projected/4758dcdb-0efd-4893-a54d-b5af9deff917-kube-api-access-l4md7\") pod \"test-model-81df64-kserve-65847c6f5-tcjt5\" (UID: \"4758dcdb-0efd-4893-a54d-b5af9deff917\") " pod="ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5"
Sep 12 00:48:10.220339 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:10.220210    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4758dcdb-0efd-4893-a54d-b5af9deff917-home\") pod \"test-model-81df64-kserve-65847c6f5-tcjt5\" (UID: \"4758dcdb-0efd-4893-a54d-b5af9deff917\") " pod="ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5"
Sep 12 00:48:10.220339 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:10.220274    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4758dcdb-0efd-4893-a54d-b5af9deff917-tmp-dir\") pod \"test-model-81df64-kserve-65847c6f5-tcjt5\" (UID: \"4758dcdb-0efd-4893-a54d-b5af9deff917\") " pod="ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5"
Sep 12 00:48:10.220339 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:10.220308    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4758dcdb-0efd-4893-a54d-b5af9deff917-tls-certs\") pod \"test-model-81df64-kserve-65847c6f5-tcjt5\" (UID: \"4758dcdb-0efd-4893-a54d-b5af9deff917\") " pod="ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5"
Sep 12 00:48:10.220339 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:10.220332    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4758dcdb-0efd-4893-a54d-b5af9deff917-dshm\") pod \"test-model-81df64-kserve-65847c6f5-tcjt5\" (UID: \"4758dcdb-0efd-4893-a54d-b5af9deff917\") " pod="ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5"
Sep 12 00:48:10.220482 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:10.220380    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4758dcdb-0efd-4893-a54d-b5af9deff917-model-cache\") pod \"test-model-81df64-kserve-65847c6f5-tcjt5\" (UID: \"4758dcdb-0efd-4893-a54d-b5af9deff917\") " pod="ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5"
Sep 12 00:48:10.321821 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:10.321777    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4758dcdb-0efd-4893-a54d-b5af9deff917-tls-certs\") pod \"test-model-81df64-kserve-65847c6f5-tcjt5\" (UID: \"4758dcdb-0efd-4893-a54d-b5af9deff917\") " pod="ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5"
Sep 12 00:48:10.321821 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:10.321822    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4758dcdb-0efd-4893-a54d-b5af9deff917-dshm\") pod \"test-model-81df64-kserve-65847c6f5-tcjt5\" (UID: \"4758dcdb-0efd-4893-a54d-b5af9deff917\") " pod="ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5"
Sep 12 00:48:10.322025 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:10.321865    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4758dcdb-0efd-4893-a54d-b5af9deff917-model-cache\") pod \"test-model-81df64-kserve-65847c6f5-tcjt5\" (UID: \"4758dcdb-0efd-4893-a54d-b5af9deff917\") " pod="ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5"
Sep 12 00:48:10.322025 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:10.321930    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4md7\" (UniqueName: \"kubernetes.io/projected/4758dcdb-0efd-4893-a54d-b5af9deff917-kube-api-access-l4md7\") pod \"test-model-81df64-kserve-65847c6f5-tcjt5\" (UID: \"4758dcdb-0efd-4893-a54d-b5af9deff917\") " pod="ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5"
Sep 12 00:48:10.322025 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:10.321980    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4758dcdb-0efd-4893-a54d-b5af9deff917-home\") pod \"test-model-81df64-kserve-65847c6f5-tcjt5\" (UID: \"4758dcdb-0efd-4893-a54d-b5af9deff917\") " pod="ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5"
Sep 12 00:48:10.322214 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:10.322026    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4758dcdb-0efd-4893-a54d-b5af9deff917-tmp-dir\") pod \"test-model-81df64-kserve-65847c6f5-tcjt5\" (UID: \"4758dcdb-0efd-4893-a54d-b5af9deff917\") " pod="ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5"
Sep 12 00:48:10.322380 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:10.322355    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4758dcdb-0efd-4893-a54d-b5af9deff917-model-cache\") pod \"test-model-81df64-kserve-65847c6f5-tcjt5\" (UID: \"4758dcdb-0efd-4893-a54d-b5af9deff917\") " pod="ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5"
Sep 12 00:48:10.322460 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:10.322425    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4758dcdb-0efd-4893-a54d-b5af9deff917-home\") pod \"test-model-81df64-kserve-65847c6f5-tcjt5\" (UID: \"4758dcdb-0efd-4893-a54d-b5af9deff917\") " pod="ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5"
Sep 12 00:48:10.322460 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:10.322440    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4758dcdb-0efd-4893-a54d-b5af9deff917-tmp-dir\") pod \"test-model-81df64-kserve-65847c6f5-tcjt5\" (UID: \"4758dcdb-0efd-4893-a54d-b5af9deff917\") " pod="ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5"
Sep 12 00:48:10.324768 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:10.324748    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4758dcdb-0efd-4893-a54d-b5af9deff917-dshm\") pod \"test-model-81df64-kserve-65847c6f5-tcjt5\" (UID: \"4758dcdb-0efd-4893-a54d-b5af9deff917\") " pod="ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5"
Sep 12 00:48:10.325168 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:10.325145    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4758dcdb-0efd-4893-a54d-b5af9deff917-tls-certs\") pod \"test-model-81df64-kserve-65847c6f5-tcjt5\" (UID: \"4758dcdb-0efd-4893-a54d-b5af9deff917\") " pod="ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5"
Sep 12 00:48:10.332385 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:10.332367    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4md7\" (UniqueName: \"kubernetes.io/projected/4758dcdb-0efd-4893-a54d-b5af9deff917-kube-api-access-l4md7\") pod \"test-model-81df64-kserve-65847c6f5-tcjt5\" (UID: \"4758dcdb-0efd-4893-a54d-b5af9deff917\") " pod="ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5"
Sep 12 00:48:10.410237 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:10.410131    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5"
Sep 12 00:48:10.604412 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:10.604382    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:10.682884 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:10.682851    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5"]
Sep 12 00:48:10.684432 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:48:10.684393    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4758dcdb_0efd_4893_a54d_b5af9deff917.slice/crio-39035228fc906c75798f1a8e7d7c2e2e5780bc158ff7a9f923e9818b578bf8cf WatchSource:0}: Error finding container 39035228fc906c75798f1a8e7d7c2e2e5780bc158ff7a9f923e9818b578bf8cf: Status 404 returned error can't find the container with id 39035228fc906c75798f1a8e7d7c2e2e5780bc158ff7a9f923e9818b578bf8cf
Sep 12 00:48:11.230547 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:11.230509    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:11.542735 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:11.542638    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5" event={"ID":"4758dcdb-0efd-4893-a54d-b5af9deff917","Type":"ContainerStarted","Data":"7e39bc597571332751a3bcae70eac74d8b2f244c0f0b3d1f8b9f01b4cb91f73d"}
Sep 12 00:48:11.542735 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:11.542678    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5" event={"ID":"4758dcdb-0efd-4893-a54d-b5af9deff917","Type":"ContainerStarted","Data":"39035228fc906c75798f1a8e7d7c2e2e5780bc158ff7a9f923e9818b578bf8cf"}
Sep 12 00:48:11.560601 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:11.560544    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5" podStartSLOduration=1.5605264669999999 podStartE2EDuration="1.560526467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:48:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:48:11.559403423 +0000 UTC m=+1128.233462283" watchObservedRunningTime="2026-09-12 00:48:11.560526467 +0000 UTC m=+1128.234585326"
Sep 12 00:48:20.410496 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:20.410451    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5"
Sep 12 00:48:20.410496 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:20.410506    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5"
Sep 12 00:48:20.423451 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:20.423426    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5"
Sep 12 00:48:20.600728 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:20.600699    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5"
Sep 12 00:48:23.686942 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:23.686890    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:24.443873 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:24.443818    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:25.077421 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:25.077390    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:25.871715 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:25.871675    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:26.687678 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:26.687639    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:27.661644 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:27.661610    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:28.367243 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:28.367203    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:28.988688 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:28.988652    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:29.779322 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:29.779289    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:30.421550 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:30.421499    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:31.168221 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:31.168185    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:32.964317 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:32.964281    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:33.576269 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:33.576232    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm"]
Sep 12 00:48:33.576516 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:33.576489    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm" podUID="7c0d2758-05db-4945-b531-73c1850e4b2c" containerName="maas-api" containerID="cri-o://2d38bd79b206d7da640ef59b24ae47829c3ae0d0bfa240c53775e4f8e6a6e230" gracePeriod=30
Sep 12 00:48:33.604762 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:33.604733    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:33.668147 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:33.668112    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:33.777235 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:33.777200    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:33.846376 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:33.846286    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:33.892251 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:33.892219    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm"
Sep 12 00:48:33.938010 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:33.937978    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:33.975503 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:33.975469    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c0d2758-05db-4945-b531-73c1850e4b2c-openshift-service-ca\") pod \"7c0d2758-05db-4945-b531-73c1850e4b2c\" (UID: \"7c0d2758-05db-4945-b531-73c1850e4b2c\") "
Sep 12 00:48:33.975983 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:33.975547    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c0d2758-05db-4945-b531-73c1850e4b2c-metrics-certs\") pod \"7c0d2758-05db-4945-b531-73c1850e4b2c\" (UID: \"7c0d2758-05db-4945-b531-73c1850e4b2c\") "
Sep 12 00:48:33.975983 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:33.975611    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9c8w\" (UniqueName: \"kubernetes.io/projected/7c0d2758-05db-4945-b531-73c1850e4b2c-kube-api-access-r9c8w\") pod \"7c0d2758-05db-4945-b531-73c1850e4b2c\" (UID: \"7c0d2758-05db-4945-b531-73c1850e4b2c\") "
Sep 12 00:48:33.975983 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:33.975768    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c0d2758-05db-4945-b531-73c1850e4b2c-maas-api-tls\") pod \"7c0d2758-05db-4945-b531-73c1850e4b2c\" (UID: \"7c0d2758-05db-4945-b531-73c1850e4b2c\") "
Sep 12 00:48:33.975983 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:33.975823    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c0d2758-05db-4945-b531-73c1850e4b2c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7c0d2758-05db-4945-b531-73c1850e4b2c" (UID: "7c0d2758-05db-4945-b531-73c1850e4b2c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:48:33.976222 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:33.976168    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c0d2758-05db-4945-b531-73c1850e4b2c-openshift-service-ca\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:48:33.977966 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:33.977936    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c0d2758-05db-4945-b531-73c1850e4b2c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7c0d2758-05db-4945-b531-73c1850e4b2c" (UID: "7c0d2758-05db-4945-b531-73c1850e4b2c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:48:33.978120 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:33.978007    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c0d2758-05db-4945-b531-73c1850e4b2c-kube-api-access-r9c8w" (OuterVolumeSpecName: "kube-api-access-r9c8w") pod "7c0d2758-05db-4945-b531-73c1850e4b2c" (UID: "7c0d2758-05db-4945-b531-73c1850e4b2c"). InnerVolumeSpecName "kube-api-access-r9c8w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:48:33.978120 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:33.978012    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c0d2758-05db-4945-b531-73c1850e4b2c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7c0d2758-05db-4945-b531-73c1850e4b2c" (UID: "7c0d2758-05db-4945-b531-73c1850e4b2c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:48:34.077425 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:34.077386    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c0d2758-05db-4945-b531-73c1850e4b2c-metrics-certs\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:48:34.077425 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:34.077417    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9c8w\" (UniqueName: \"kubernetes.io/projected/7c0d2758-05db-4945-b531-73c1850e4b2c-kube-api-access-r9c8w\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:48:34.077425 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:34.077430    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c0d2758-05db-4945-b531-73c1850e4b2c-maas-api-tls\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:48:34.135059 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:34.134964    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:34.193710 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:34.193678    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:34.278211 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:34.278182    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:34.379700 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:34.379666    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:34.666005 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:34.665967    2558 generic.go:356] "Generic (PLEG): container finished" podID="7c0d2758-05db-4945-b531-73c1850e4b2c" containerID="2d38bd79b206d7da640ef59b24ae47829c3ae0d0bfa240c53775e4f8e6a6e230" exitCode=0
Sep 12 00:48:34.666223 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:34.666020    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm" event={"ID":"7c0d2758-05db-4945-b531-73c1850e4b2c","Type":"ContainerDied","Data":"2d38bd79b206d7da640ef59b24ae47829c3ae0d0bfa240c53775e4f8e6a6e230"}
Sep 12 00:48:34.666223 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:34.666027    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm"
Sep 12 00:48:34.666223 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:34.666075    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm" event={"ID":"7c0d2758-05db-4945-b531-73c1850e4b2c","Type":"ContainerDied","Data":"a33bb1f0ef6669f5d814025ff8737ebb49a852071215459974b9ad28b2ed921b"}
Sep 12 00:48:34.666223 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:34.666135    2558 scope.go:117] "RemoveContainer" containerID="2d38bd79b206d7da640ef59b24ae47829c3ae0d0bfa240c53775e4f8e6a6e230"
Sep 12 00:48:34.676996 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:34.676974    2558 scope.go:117] "RemoveContainer" containerID="2d38bd79b206d7da640ef59b24ae47829c3ae0d0bfa240c53775e4f8e6a6e230"
Sep 12 00:48:34.677344 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:48:34.677316    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d38bd79b206d7da640ef59b24ae47829c3ae0d0bfa240c53775e4f8e6a6e230\": container with ID starting with 2d38bd79b206d7da640ef59b24ae47829c3ae0d0bfa240c53775e4f8e6a6e230 not found: ID does not exist" containerID="2d38bd79b206d7da640ef59b24ae47829c3ae0d0bfa240c53775e4f8e6a6e230"
Sep 12 00:48:34.677410 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:34.677354    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d38bd79b206d7da640ef59b24ae47829c3ae0d0bfa240c53775e4f8e6a6e230"} err="failed to get container status \"2d38bd79b206d7da640ef59b24ae47829c3ae0d0bfa240c53775e4f8e6a6e230\": rpc error: code = NotFound desc = could not find container \"2d38bd79b206d7da640ef59b24ae47829c3ae0d0bfa240c53775e4f8e6a6e230\": container with ID starting with 2d38bd79b206d7da640ef59b24ae47829c3ae0d0bfa240c53775e4f8e6a6e230 not found: ID does not exist"
Sep 12 00:48:34.688229 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:34.688207    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm"]
Sep 12 00:48:34.693895 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:34.693875    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-4b6ed7-75c5658cd9-x76vm"]
Sep 12 00:48:34.902211 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:34.902174    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:34.999351 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:34.999316    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:35.101671 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:35.101632    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:35.357472 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:35.357396    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:35.738902 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:35.738867    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c0d2758-05db-4945-b531-73c1850e4b2c" path="/var/lib/kubelet/pods/7c0d2758-05db-4945-b531-73c1850e4b2c/volumes"
Sep 12 00:48:36.013448 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:36.013361    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:36.164277 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:36.164248    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:36.407442 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:36.407357    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:36.890376 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:36.890343    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:37.548584 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:37.548508    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:38.177458 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:38.177426    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:39.131543 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:39.131497    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f"]
Sep 12 00:48:39.182701 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:39.182670    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:42.262738 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:42.262697    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:42.367882 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:42.367850    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:42.462826 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:42.462782    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:42.587226 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:42.587134    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:42.922811 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:42.922731    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:48.974733 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:48.974653    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:48:54.151290 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:54.151221    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f" podUID="e68f3ff0-dc53-4295-b634-44e0a6bb272f" containerName="main" containerID="cri-o://bab816f15c2c9985eb57547102f12f8bce35f1075e36d7a44e3acb20bc98526f" gracePeriod=45
Sep 12 00:48:54.411455 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:54.411395    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f"
Sep 12 00:48:54.488484 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:54.488436    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e68f3ff0-dc53-4295-b634-44e0a6bb272f-model-cache\") pod \"e68f3ff0-dc53-4295-b634-44e0a6bb272f\" (UID: \"e68f3ff0-dc53-4295-b634-44e0a6bb272f\") "
Sep 12 00:48:54.488484 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:54.488490    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l92fw\" (UniqueName: \"kubernetes.io/projected/e68f3ff0-dc53-4295-b634-44e0a6bb272f-kube-api-access-l92fw\") pod \"e68f3ff0-dc53-4295-b634-44e0a6bb272f\" (UID: \"e68f3ff0-dc53-4295-b634-44e0a6bb272f\") "
Sep 12 00:48:54.488712 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:54.488541    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e68f3ff0-dc53-4295-b634-44e0a6bb272f-tmp-dir\") pod \"e68f3ff0-dc53-4295-b634-44e0a6bb272f\" (UID: \"e68f3ff0-dc53-4295-b634-44e0a6bb272f\") "
Sep 12 00:48:54.488712 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:54.488562    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e68f3ff0-dc53-4295-b634-44e0a6bb272f-tls-certs\") pod \"e68f3ff0-dc53-4295-b634-44e0a6bb272f\" (UID: \"e68f3ff0-dc53-4295-b634-44e0a6bb272f\") "
Sep 12 00:48:54.488712 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:54.488633    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e68f3ff0-dc53-4295-b634-44e0a6bb272f-home\") pod \"e68f3ff0-dc53-4295-b634-44e0a6bb272f\" (UID: \"e68f3ff0-dc53-4295-b634-44e0a6bb272f\") "
Sep 12 00:48:54.488712 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:54.488695    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e68f3ff0-dc53-4295-b634-44e0a6bb272f-dshm\") pod \"e68f3ff0-dc53-4295-b634-44e0a6bb272f\" (UID: \"e68f3ff0-dc53-4295-b634-44e0a6bb272f\") "
Sep 12 00:48:54.488864 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:54.488761    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e68f3ff0-dc53-4295-b634-44e0a6bb272f-model-cache" (OuterVolumeSpecName: "model-cache") pod "e68f3ff0-dc53-4295-b634-44e0a6bb272f" (UID: "e68f3ff0-dc53-4295-b634-44e0a6bb272f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:48:54.488928 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:54.488862    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e68f3ff0-dc53-4295-b634-44e0a6bb272f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e68f3ff0-dc53-4295-b634-44e0a6bb272f" (UID: "e68f3ff0-dc53-4295-b634-44e0a6bb272f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:48:54.489032 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:54.489015    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e68f3ff0-dc53-4295-b634-44e0a6bb272f-model-cache\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:48:54.489112 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:54.489023    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e68f3ff0-dc53-4295-b634-44e0a6bb272f-home" (OuterVolumeSpecName: "home") pod "e68f3ff0-dc53-4295-b634-44e0a6bb272f" (UID: "e68f3ff0-dc53-4295-b634-44e0a6bb272f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:48:54.489112 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:54.489040    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e68f3ff0-dc53-4295-b634-44e0a6bb272f-tmp-dir\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:48:54.490939 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:54.490913    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e68f3ff0-dc53-4295-b634-44e0a6bb272f-dshm" (OuterVolumeSpecName: "dshm") pod "e68f3ff0-dc53-4295-b634-44e0a6bb272f" (UID: "e68f3ff0-dc53-4295-b634-44e0a6bb272f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:48:54.491069 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:54.490968    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e68f3ff0-dc53-4295-b634-44e0a6bb272f-kube-api-access-l92fw" (OuterVolumeSpecName: "kube-api-access-l92fw") pod "e68f3ff0-dc53-4295-b634-44e0a6bb272f" (UID: "e68f3ff0-dc53-4295-b634-44e0a6bb272f"). InnerVolumeSpecName "kube-api-access-l92fw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:48:54.491184 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:54.491076    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e68f3ff0-dc53-4295-b634-44e0a6bb272f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e68f3ff0-dc53-4295-b634-44e0a6bb272f" (UID: "e68f3ff0-dc53-4295-b634-44e0a6bb272f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:48:54.589936 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:54.589884    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e68f3ff0-dc53-4295-b634-44e0a6bb272f-tls-certs\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:48:54.589936 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:54.589929    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e68f3ff0-dc53-4295-b634-44e0a6bb272f-home\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:48:54.589936 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:54.589938    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e68f3ff0-dc53-4295-b634-44e0a6bb272f-dshm\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:48:54.589936 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:54.589947    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l92fw\" (UniqueName: \"kubernetes.io/projected/e68f3ff0-dc53-4295-b634-44e0a6bb272f-kube-api-access-l92fw\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:48:54.774466 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:54.774433    2558 generic.go:356] "Generic (PLEG): container finished" podID="e68f3ff0-dc53-4295-b634-44e0a6bb272f" containerID="bab816f15c2c9985eb57547102f12f8bce35f1075e36d7a44e3acb20bc98526f" exitCode=0
Sep 12 00:48:54.774682 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:54.774480    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f"
Sep 12 00:48:54.774682 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:54.774521    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f" event={"ID":"e68f3ff0-dc53-4295-b634-44e0a6bb272f","Type":"ContainerDied","Data":"bab816f15c2c9985eb57547102f12f8bce35f1075e36d7a44e3acb20bc98526f"}
Sep 12 00:48:54.774682 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:54.774572    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f" event={"ID":"e68f3ff0-dc53-4295-b634-44e0a6bb272f","Type":"ContainerDied","Data":"f691ca15653e7fe73e43d61b3d0088232d11f0e1ec9e0f3eb7e0e3cb572fc9ca"}
Sep 12 00:48:54.774682 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:54.774595    2558 scope.go:117] "RemoveContainer" containerID="bab816f15c2c9985eb57547102f12f8bce35f1075e36d7a44e3acb20bc98526f"
Sep 12 00:48:54.784509 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:54.784492    2558 scope.go:117] "RemoveContainer" containerID="bab816f15c2c9985eb57547102f12f8bce35f1075e36d7a44e3acb20bc98526f"
Sep 12 00:48:54.784773 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:48:54.784756    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bab816f15c2c9985eb57547102f12f8bce35f1075e36d7a44e3acb20bc98526f\": container with ID starting with bab816f15c2c9985eb57547102f12f8bce35f1075e36d7a44e3acb20bc98526f not found: ID does not exist" containerID="bab816f15c2c9985eb57547102f12f8bce35f1075e36d7a44e3acb20bc98526f"
Sep 12 00:48:54.784821 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:54.784782    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bab816f15c2c9985eb57547102f12f8bce35f1075e36d7a44e3acb20bc98526f"} err="failed to get container status \"bab816f15c2c9985eb57547102f12f8bce35f1075e36d7a44e3acb20bc98526f\": rpc error: code = NotFound desc = could not find container \"bab816f15c2c9985eb57547102f12f8bce35f1075e36d7a44e3acb20bc98526f\": container with ID starting with bab816f15c2c9985eb57547102f12f8bce35f1075e36d7a44e3acb20bc98526f not found: ID does not exist"
Sep 12 00:48:54.795225 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:54.795202    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f"]
Sep 12 00:48:54.798748 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:54.798723    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-4b6ed7/autores-model-4b6ed7-kserve-6f64967cf9-8n29f"]
Sep 12 00:48:55.739289 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:48:55.739255    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e68f3ff0-dc53-4295-b634-44e0a6bb272f" path="/var/lib/kubelet/pods/e68f3ff0-dc53-4295-b634-44e0a6bb272f/volumes"
Sep 12 00:49:01.929389 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:01.929349    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:03.646342 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:03.646309    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:03.896807 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:03.896707    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:03.993775 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:03.993745    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:04.344861 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:04.344823    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:04.661841 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:04.661733    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:05.274579 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:05.274542    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:07.352611 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:07.352569    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:07.514350 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:07.514314    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:07.612138 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:07.612011    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:08.280992 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:08.280961    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:08.825212 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:08.825176    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:09.007398 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:09.007360    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:09.812701 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:09.812669    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:11.132384 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:11.132339    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:11.307652 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:11.307617    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:11.566295 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:11.566266    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:11.775334 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:11.775296    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:11.879733 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:11.879643    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:14.951593 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:14.951564    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:15.061688 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:15.061659    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:15.153296 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:15.153267    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:23.791067 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:23.790950    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-czf8h_dd90996a-5428-4ccf-a154-c0aa8def9789/console-operator/1.log"
Sep 12 00:49:23.795150 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:23.791384    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-czf8h_dd90996a-5428-4ccf-a154-c0aa8def9789/console-operator/1.log"
Sep 12 00:49:25.705623 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:25.705581    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:30.959391 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:30.959347    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:33.479382 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:33.479327    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:34.402309 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:34.402265    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:35.211236 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:35.211196    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:36.035820 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:36.035768    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:36.724148 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:36.724106    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:37.365311 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:37.365276    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:37.565147 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:37.565103    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:37.662293 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:37.662194    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:37.761855 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:37.761803    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:37.861730 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:37.861696    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:42.929993 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:42.929961    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2"]
Sep 12 00:49:42.930609 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:42.930587    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e68f3ff0-dc53-4295-b634-44e0a6bb272f" containerName="main"
Sep 12 00:49:42.930694 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:42.930612    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e68f3ff0-dc53-4295-b634-44e0a6bb272f" containerName="main"
Sep 12 00:49:42.930694 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:42.930636    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c0d2758-05db-4945-b531-73c1850e4b2c" containerName="maas-api"
Sep 12 00:49:42.930694 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:42.930645    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c0d2758-05db-4945-b531-73c1850e4b2c" containerName="maas-api"
Sep 12 00:49:42.930839 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:42.930779    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e68f3ff0-dc53-4295-b634-44e0a6bb272f" containerName="main"
Sep 12 00:49:42.930839 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:42.930800    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c0d2758-05db-4945-b531-73c1850e4b2c" containerName="maas-api"
Sep 12 00:49:42.934430 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:42.934408    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2"
Sep 12 00:49:42.937331 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:42.937311    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-bb6e74\"/\"openshift-service-ca.crt\""
Sep 12 00:49:42.938318 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:42.938283    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-bb6e74\"/\"default-dockercfg-97xk9\""
Sep 12 00:49:42.938453 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:42.938346    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-bb6e74\"/\"kube-root-ca.crt\""
Sep 12 00:49:42.938453 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:42.938346    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-bb6e74\"/\"test-model-bb6e74-kserve-self-signed-certs\""
Sep 12 00:49:42.945670 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:42.945649    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2"]
Sep 12 00:49:42.968054 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:42.968026    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:43.064168 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:43.064131    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81b591cf-5ed8-4327-91a0-52e95b76c971-tls-certs\") pod \"test-model-bb6e74-kserve-6996fcf55c-c87x2\" (UID: \"81b591cf-5ed8-4327-91a0-52e95b76c971\") " pod="ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2"
Sep 12 00:49:43.064339 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:43.064196    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81b591cf-5ed8-4327-91a0-52e95b76c971-home\") pod \"test-model-bb6e74-kserve-6996fcf55c-c87x2\" (UID: \"81b591cf-5ed8-4327-91a0-52e95b76c971\") " pod="ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2"
Sep 12 00:49:43.064339 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:43.064219    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t89b\" (UniqueName: \"kubernetes.io/projected/81b591cf-5ed8-4327-91a0-52e95b76c971-kube-api-access-6t89b\") pod \"test-model-bb6e74-kserve-6996fcf55c-c87x2\" (UID: \"81b591cf-5ed8-4327-91a0-52e95b76c971\") " pod="ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2"
Sep 12 00:49:43.064339 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:43.064262    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81b591cf-5ed8-4327-91a0-52e95b76c971-tmp-dir\") pod \"test-model-bb6e74-kserve-6996fcf55c-c87x2\" (UID: \"81b591cf-5ed8-4327-91a0-52e95b76c971\") " pod="ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2"
Sep 12 00:49:43.064450 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:43.064358    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81b591cf-5ed8-4327-91a0-52e95b76c971-model-cache\") pod \"test-model-bb6e74-kserve-6996fcf55c-c87x2\" (UID: \"81b591cf-5ed8-4327-91a0-52e95b76c971\") " pod="ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2"
Sep 12 00:49:43.064450 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:43.064403    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81b591cf-5ed8-4327-91a0-52e95b76c971-dshm\") pod \"test-model-bb6e74-kserve-6996fcf55c-c87x2\" (UID: \"81b591cf-5ed8-4327-91a0-52e95b76c971\") " pod="ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2"
Sep 12 00:49:43.165405 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:43.165368    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81b591cf-5ed8-4327-91a0-52e95b76c971-dshm\") pod \"test-model-bb6e74-kserve-6996fcf55c-c87x2\" (UID: \"81b591cf-5ed8-4327-91a0-52e95b76c971\") " pod="ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2"
Sep 12 00:49:43.165405 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:43.165406    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81b591cf-5ed8-4327-91a0-52e95b76c971-tls-certs\") pod \"test-model-bb6e74-kserve-6996fcf55c-c87x2\" (UID: \"81b591cf-5ed8-4327-91a0-52e95b76c971\") " pod="ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2"
Sep 12 00:49:43.165647 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:43.165551    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81b591cf-5ed8-4327-91a0-52e95b76c971-home\") pod \"test-model-bb6e74-kserve-6996fcf55c-c87x2\" (UID: \"81b591cf-5ed8-4327-91a0-52e95b76c971\") " pod="ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2"
Sep 12 00:49:43.165647 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:43.165603    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t89b\" (UniqueName: \"kubernetes.io/projected/81b591cf-5ed8-4327-91a0-52e95b76c971-kube-api-access-6t89b\") pod \"test-model-bb6e74-kserve-6996fcf55c-c87x2\" (UID: \"81b591cf-5ed8-4327-91a0-52e95b76c971\") " pod="ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2"
Sep 12 00:49:43.165747 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:43.165677    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81b591cf-5ed8-4327-91a0-52e95b76c971-tmp-dir\") pod \"test-model-bb6e74-kserve-6996fcf55c-c87x2\" (UID: \"81b591cf-5ed8-4327-91a0-52e95b76c971\") " pod="ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2"
Sep 12 00:49:43.165797 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:43.165750    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81b591cf-5ed8-4327-91a0-52e95b76c971-model-cache\") pod \"test-model-bb6e74-kserve-6996fcf55c-c87x2\" (UID: \"81b591cf-5ed8-4327-91a0-52e95b76c971\") " pod="ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2"
Sep 12 00:49:43.165911 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:43.165888    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81b591cf-5ed8-4327-91a0-52e95b76c971-home\") pod \"test-model-bb6e74-kserve-6996fcf55c-c87x2\" (UID: \"81b591cf-5ed8-4327-91a0-52e95b76c971\") " pod="ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2"
Sep 12 00:49:43.166123 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:43.166078    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81b591cf-5ed8-4327-91a0-52e95b76c971-tmp-dir\") pod \"test-model-bb6e74-kserve-6996fcf55c-c87x2\" (UID: \"81b591cf-5ed8-4327-91a0-52e95b76c971\") " pod="ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2"
Sep 12 00:49:43.166211 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:43.166173    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81b591cf-5ed8-4327-91a0-52e95b76c971-model-cache\") pod \"test-model-bb6e74-kserve-6996fcf55c-c87x2\" (UID: \"81b591cf-5ed8-4327-91a0-52e95b76c971\") " pod="ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2"
Sep 12 00:49:43.167919 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:43.167898    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81b591cf-5ed8-4327-91a0-52e95b76c971-dshm\") pod \"test-model-bb6e74-kserve-6996fcf55c-c87x2\" (UID: \"81b591cf-5ed8-4327-91a0-52e95b76c971\") " pod="ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2"
Sep 12 00:49:43.168053 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:43.168038    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81b591cf-5ed8-4327-91a0-52e95b76c971-tls-certs\") pod \"test-model-bb6e74-kserve-6996fcf55c-c87x2\" (UID: \"81b591cf-5ed8-4327-91a0-52e95b76c971\") " pod="ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2"
Sep 12 00:49:43.175142 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:43.175047    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t89b\" (UniqueName: \"kubernetes.io/projected/81b591cf-5ed8-4327-91a0-52e95b76c971-kube-api-access-6t89b\") pod \"test-model-bb6e74-kserve-6996fcf55c-c87x2\" (UID: \"81b591cf-5ed8-4327-91a0-52e95b76c971\") " pod="ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2"
Sep 12 00:49:43.247367 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:43.247325    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2"
Sep 12 00:49:43.527686 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:43.527639    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2"]
Sep 12 00:49:43.533760 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:49:43.532788    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod81b591cf_5ed8_4327_91a0_52e95b76c971.slice/crio-1f330b916c2ef2ef199c49724f673454318aed98f315066c2e7c9ba7f05783f4 WatchSource:0}: Error finding container 1f330b916c2ef2ef199c49724f673454318aed98f315066c2e7c9ba7f05783f4: Status 404 returned error can't find the container with id 1f330b916c2ef2ef199c49724f673454318aed98f315066c2e7c9ba7f05783f4
Sep 12 00:49:44.036665 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:44.036616    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2" event={"ID":"81b591cf-5ed8-4327-91a0-52e95b76c971","Type":"ContainerStarted","Data":"d61242502e7bd8ef795b43853004221779bb031866146c4dde2977a1924b0faa"}
Sep 12 00:49:44.037171 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:44.036670    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2" event={"ID":"81b591cf-5ed8-4327-91a0-52e95b76c971","Type":"ContainerStarted","Data":"1f330b916c2ef2ef199c49724f673454318aed98f315066c2e7c9ba7f05783f4"}
Sep 12 00:49:44.057635 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:44.057584    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2" podStartSLOduration=2.057570049 podStartE2EDuration="2.057570049s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:49:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:49:44.056188422 +0000 UTC m=+1220.730247293" watchObservedRunningTime="2026-09-12 00:49:44.057570049 +0000 UTC m=+1220.731628907"
Sep 12 00:49:48.428635 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:48.428599    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:49.170700 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:49.170665    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:53.247966 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:53.247913    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2"
Sep 12 00:49:53.248414 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:53.247989    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2"
Sep 12 00:49:53.260871 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:53.260837    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2"
Sep 12 00:49:54.097309 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:54.097280    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2"
Sep 12 00:49:54.914790 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:54.914754    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:55.021573 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:55.021540    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:55.115128 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:55.115069    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:55.215134 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:55.215078    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:55.811226 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:55.811192    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:55.925405 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:55.925374    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:56.062882 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:56.062797    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:49:57.426007 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:49:57.425969    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:03.224892 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:03.224860    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:03.302505 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:03.302477    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:03.881212 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:03.881175    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:09.724632 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:09.724591    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:09.823709 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:09.823670    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:09.950740 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:09.950708    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:10.258063 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:10.258022    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:10.436354 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:10.436317    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:11.517727 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:11.517694    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:11.618583 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:11.618550    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:16.679492 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:16.679418    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:16.926209 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:16.926175    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:17.034762 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:17.034729    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:17.697576 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:17.697542    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:18.782779 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:18.782745    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:19.650470 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:19.650438    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:20.479639 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:20.479603    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:21.166665 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:21.166631    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf"]
Sep 12 00:50:21.171239 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:21.171216    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf"
Sep 12 00:50:21.173921 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:21.173893    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-ee23f934-serving-cert\""
Sep 12 00:50:21.182669 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:21.182645    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf"]
Sep 12 00:50:21.216264 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:21.216237    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:21.261050 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:21.261019    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d110ac11-1d50-4d03-963f-f8f93f722081-openshift-service-ca\") pod \"maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf\" (UID: \"d110ac11-1d50-4d03-963f-f8f93f722081\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf"
Sep 12 00:50:21.261230 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:21.261061    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnt5f\" (UniqueName: \"kubernetes.io/projected/d110ac11-1d50-4d03-963f-f8f93f722081-kube-api-access-mnt5f\") pod \"maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf\" (UID: \"d110ac11-1d50-4d03-963f-f8f93f722081\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf"
Sep 12 00:50:21.261230 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:21.261116    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d110ac11-1d50-4d03-963f-f8f93f722081-metrics-certs\") pod \"maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf\" (UID: \"d110ac11-1d50-4d03-963f-f8f93f722081\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf"
Sep 12 00:50:21.261230 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:21.261163    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d110ac11-1d50-4d03-963f-f8f93f722081-maas-api-tls\") pod \"maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf\" (UID: \"d110ac11-1d50-4d03-963f-f8f93f722081\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf"
Sep 12 00:50:21.312198 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:21.312169    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:21.362376 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:21.362338    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d110ac11-1d50-4d03-963f-f8f93f722081-openshift-service-ca\") pod \"maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf\" (UID: \"d110ac11-1d50-4d03-963f-f8f93f722081\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf"
Sep 12 00:50:21.362565 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:21.362387    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnt5f\" (UniqueName: \"kubernetes.io/projected/d110ac11-1d50-4d03-963f-f8f93f722081-kube-api-access-mnt5f\") pod \"maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf\" (UID: \"d110ac11-1d50-4d03-963f-f8f93f722081\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf"
Sep 12 00:50:21.362565 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:21.362423    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d110ac11-1d50-4d03-963f-f8f93f722081-metrics-certs\") pod \"maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf\" (UID: \"d110ac11-1d50-4d03-963f-f8f93f722081\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf"
Sep 12 00:50:21.362565 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:21.362447    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d110ac11-1d50-4d03-963f-f8f93f722081-maas-api-tls\") pod \"maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf\" (UID: \"d110ac11-1d50-4d03-963f-f8f93f722081\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf"
Sep 12 00:50:21.362993 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:21.362971    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d110ac11-1d50-4d03-963f-f8f93f722081-openshift-service-ca\") pod \"maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf\" (UID: \"d110ac11-1d50-4d03-963f-f8f93f722081\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf"
Sep 12 00:50:21.365052 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:21.365026    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d110ac11-1d50-4d03-963f-f8f93f722081-maas-api-tls\") pod \"maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf\" (UID: \"d110ac11-1d50-4d03-963f-f8f93f722081\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf"
Sep 12 00:50:21.365175 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:21.365155    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d110ac11-1d50-4d03-963f-f8f93f722081-metrics-certs\") pod \"maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf\" (UID: \"d110ac11-1d50-4d03-963f-f8f93f722081\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf"
Sep 12 00:50:21.372180 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:21.372155    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnt5f\" (UniqueName: \"kubernetes.io/projected/d110ac11-1d50-4d03-963f-f8f93f722081-kube-api-access-mnt5f\") pod \"maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf\" (UID: \"d110ac11-1d50-4d03-963f-f8f93f722081\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf"
Sep 12 00:50:21.409643 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:21.409615    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:21.483268 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:21.483242    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf"
Sep 12 00:50:21.775077 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:21.768237    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf"]
Sep 12 00:50:21.779798 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:21.779774    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 12 00:50:21.834511 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:21.834477    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:21.934813 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:21.934779    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:22.242054 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:22.242003    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf" event={"ID":"d110ac11-1d50-4d03-963f-f8f93f722081","Type":"ContainerStarted","Data":"56f6f07b9b7d3111a4053f33d24c2e179e3760fc189808a62bdbbcc316f850a9"}
Sep 12 00:50:22.242054 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:22.242060    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf" event={"ID":"d110ac11-1d50-4d03-963f-f8f93f722081","Type":"ContainerStarted","Data":"19739555a920fc374d53b3af570b581a23c8752037b61d68e8927e2e55256bcb"}
Sep 12 00:50:22.242324 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:22.242212    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf"
Sep 12 00:50:22.265969 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:22.265913    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf" podStartSLOduration=0.927564344 podStartE2EDuration="1.26589207s" totalImagesPullingTime="338.327726ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:50:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:50:22.265234296 +0000 UTC m=+1258.939293166" watchObservedRunningTime="2026-09-12 00:50:22.26589207 +0000 UTC m=+1258.939950929"
Sep 12 00:50:28.252590 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:28.252561    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf"
Sep 12 00:50:43.446739 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:43.446701    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:44.106465 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:44.106431    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:44.866147 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:44.866107    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:45.555461 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:45.555401    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:46.325411 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:46.325374    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:46.875372 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:46.875333    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:47.220170 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:47.220133    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:47.513926 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:47.513837    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:47.614073 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:47.614038    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:47.937236 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:47.937199    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:48.106453 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:48.106414    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:49.942779 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:49.942748    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf"]
Sep 12 00:50:49.947035 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:49.947008    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf"
Sep 12 00:50:49.950067 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:49.950039    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-c3f8d8\"/\"openshift-service-ca.crt\""
Sep 12 00:50:49.950221 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:49.950068    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-c3f8d8\"/\"kube-root-ca.crt\""
Sep 12 00:50:49.951111 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:49.951071    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-c3f8d8\"/\"default-dockercfg-65pp6\""
Sep 12 00:50:49.951500 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:49.951482    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-c3f8d8\"/\"explicit-model-c3f8d8-kserve-self-signed-certs\""
Sep 12 00:50:49.967845 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:49.967821    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf"]
Sep 12 00:50:50.016998 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:50.016972    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:50.062363 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:50.062328    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kgtm\" (UniqueName: \"kubernetes.io/projected/b2566be6-5ba9-4718-9b0b-947cb094ba90-kube-api-access-7kgtm\") pod \"explicit-model-c3f8d8-kserve-84dc698d45-xc8xf\" (UID: \"b2566be6-5ba9-4718-9b0b-947cb094ba90\") " pod="ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf"
Sep 12 00:50:50.062527 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:50.062378    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2566be6-5ba9-4718-9b0b-947cb094ba90-model-cache\") pod \"explicit-model-c3f8d8-kserve-84dc698d45-xc8xf\" (UID: \"b2566be6-5ba9-4718-9b0b-947cb094ba90\") " pod="ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf"
Sep 12 00:50:50.062527 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:50.062469    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2566be6-5ba9-4718-9b0b-947cb094ba90-tmp-dir\") pod \"explicit-model-c3f8d8-kserve-84dc698d45-xc8xf\" (UID: \"b2566be6-5ba9-4718-9b0b-947cb094ba90\") " pod="ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf"
Sep 12 00:50:50.062628 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:50.062536    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2566be6-5ba9-4718-9b0b-947cb094ba90-dshm\") pod \"explicit-model-c3f8d8-kserve-84dc698d45-xc8xf\" (UID: \"b2566be6-5ba9-4718-9b0b-947cb094ba90\") " pod="ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf"
Sep 12 00:50:50.062628 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:50.062563    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2566be6-5ba9-4718-9b0b-947cb094ba90-tls-certs\") pod \"explicit-model-c3f8d8-kserve-84dc698d45-xc8xf\" (UID: \"b2566be6-5ba9-4718-9b0b-947cb094ba90\") " pod="ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf"
Sep 12 00:50:50.062628 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:50.062619    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2566be6-5ba9-4718-9b0b-947cb094ba90-home\") pod \"explicit-model-c3f8d8-kserve-84dc698d45-xc8xf\" (UID: \"b2566be6-5ba9-4718-9b0b-947cb094ba90\") " pod="ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf"
Sep 12 00:50:50.163997 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:50.163960    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2566be6-5ba9-4718-9b0b-947cb094ba90-dshm\") pod \"explicit-model-c3f8d8-kserve-84dc698d45-xc8xf\" (UID: \"b2566be6-5ba9-4718-9b0b-947cb094ba90\") " pod="ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf"
Sep 12 00:50:50.163997 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:50.163998    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2566be6-5ba9-4718-9b0b-947cb094ba90-tls-certs\") pod \"explicit-model-c3f8d8-kserve-84dc698d45-xc8xf\" (UID: \"b2566be6-5ba9-4718-9b0b-947cb094ba90\") " pod="ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf"
Sep 12 00:50:50.164240 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:50.164036    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2566be6-5ba9-4718-9b0b-947cb094ba90-home\") pod \"explicit-model-c3f8d8-kserve-84dc698d45-xc8xf\" (UID: \"b2566be6-5ba9-4718-9b0b-947cb094ba90\") " pod="ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf"
Sep 12 00:50:50.164240 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:50.164076    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kgtm\" (UniqueName: \"kubernetes.io/projected/b2566be6-5ba9-4718-9b0b-947cb094ba90-kube-api-access-7kgtm\") pod \"explicit-model-c3f8d8-kserve-84dc698d45-xc8xf\" (UID: \"b2566be6-5ba9-4718-9b0b-947cb094ba90\") " pod="ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf"
Sep 12 00:50:50.164240 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:50.164148    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2566be6-5ba9-4718-9b0b-947cb094ba90-model-cache\") pod \"explicit-model-c3f8d8-kserve-84dc698d45-xc8xf\" (UID: \"b2566be6-5ba9-4718-9b0b-947cb094ba90\") " pod="ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf"
Sep 12 00:50:50.164240 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:50.164201    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2566be6-5ba9-4718-9b0b-947cb094ba90-tmp-dir\") pod \"explicit-model-c3f8d8-kserve-84dc698d45-xc8xf\" (UID: \"b2566be6-5ba9-4718-9b0b-947cb094ba90\") " pod="ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf"
Sep 12 00:50:50.164536 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:50.164510    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2566be6-5ba9-4718-9b0b-947cb094ba90-home\") pod \"explicit-model-c3f8d8-kserve-84dc698d45-xc8xf\" (UID: \"b2566be6-5ba9-4718-9b0b-947cb094ba90\") " pod="ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf"
Sep 12 00:50:50.164616 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:50.164538    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2566be6-5ba9-4718-9b0b-947cb094ba90-model-cache\") pod \"explicit-model-c3f8d8-kserve-84dc698d45-xc8xf\" (UID: \"b2566be6-5ba9-4718-9b0b-947cb094ba90\") " pod="ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf"
Sep 12 00:50:50.164616 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:50.164558    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2566be6-5ba9-4718-9b0b-947cb094ba90-tmp-dir\") pod \"explicit-model-c3f8d8-kserve-84dc698d45-xc8xf\" (UID: \"b2566be6-5ba9-4718-9b0b-947cb094ba90\") " pod="ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf"
Sep 12 00:50:50.166510 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:50.166480    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2566be6-5ba9-4718-9b0b-947cb094ba90-dshm\") pod \"explicit-model-c3f8d8-kserve-84dc698d45-xc8xf\" (UID: \"b2566be6-5ba9-4718-9b0b-947cb094ba90\") " pod="ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf"
Sep 12 00:50:50.166667 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:50.166577    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2566be6-5ba9-4718-9b0b-947cb094ba90-tls-certs\") pod \"explicit-model-c3f8d8-kserve-84dc698d45-xc8xf\" (UID: \"b2566be6-5ba9-4718-9b0b-947cb094ba90\") " pod="ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf"
Sep 12 00:50:50.174208 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:50.174183    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kgtm\" (UniqueName: \"kubernetes.io/projected/b2566be6-5ba9-4718-9b0b-947cb094ba90-kube-api-access-7kgtm\") pod \"explicit-model-c3f8d8-kserve-84dc698d45-xc8xf\" (UID: \"b2566be6-5ba9-4718-9b0b-947cb094ba90\") " pod="ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf"
Sep 12 00:50:50.257317 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:50.257229    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf"
Sep 12 00:50:50.543901 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:50.543871    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf"]
Sep 12 00:50:50.545986 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:50:50.545951    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb2566be6_5ba9_4718_9b0b_947cb094ba90.slice/crio-efc1041287f9ea811977c760001f1fa0f9310528b97591e6193ee25561c8c6c3 WatchSource:0}: Error finding container efc1041287f9ea811977c760001f1fa0f9310528b97591e6193ee25561c8c6c3: Status 404 returned error can't find the container with id efc1041287f9ea811977c760001f1fa0f9310528b97591e6193ee25561c8c6c3
Sep 12 00:50:51.400639 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:51.400594    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf" event={"ID":"b2566be6-5ba9-4718-9b0b-947cb094ba90","Type":"ContainerStarted","Data":"a255307ee1d6a0244bb4e9bfc95515ba0d988022c92a3c3aedc87ec554bc08da"}
Sep 12 00:50:51.401140 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:51.400646    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf" event={"ID":"b2566be6-5ba9-4718-9b0b-947cb094ba90","Type":"ContainerStarted","Data":"efc1041287f9ea811977c760001f1fa0f9310528b97591e6193ee25561c8c6c3"}
Sep 12 00:50:51.431926 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:51.431874    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf" podStartSLOduration=2.431859039 podStartE2EDuration="2.431859039s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:50:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:50:51.43043009 +0000 UTC m=+1288.104488949" watchObservedRunningTime="2026-09-12 00:50:51.431859039 +0000 UTC m=+1288.105917896"
Sep 12 00:50:54.880327 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:54.880282    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:54.994788 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:54.994757    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:56.290951 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:56.290913    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:57.214520 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:57.214489    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:58.018139 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:58.018068    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:59.016383 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:59.016345    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:50:59.751337 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:50:59.751285    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:00.258226 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:00.258192    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf"
Sep 12 00:51:00.258428 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:00.258241    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf"
Sep 12 00:51:00.271563 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:00.271536    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf"
Sep 12 00:51:00.458857 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:00.458824    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf"
Sep 12 00:51:00.632557 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:00.632474    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:01.517497 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:01.517468    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:06.109137 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:06.109075    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf"]
Sep 12 00:51:06.109558 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:06.109304    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf" podUID="d110ac11-1d50-4d03-963f-f8f93f722081" containerName="maas-api" containerID="cri-o://56f6f07b9b7d3111a4053f33d24c2e179e3760fc189808a62bdbbcc316f850a9" gracePeriod=30
Sep 12 00:51:06.207038 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:06.206995    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:06.351464 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:06.351430    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:06.377267 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:06.377237    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf"
Sep 12 00:51:06.458571 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:06.458516    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:06.479657 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:06.479620    2558 generic.go:356] "Generic (PLEG): container finished" podID="d110ac11-1d50-4d03-963f-f8f93f722081" containerID="56f6f07b9b7d3111a4053f33d24c2e179e3760fc189808a62bdbbcc316f850a9" exitCode=0
Sep 12 00:51:06.479893 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:06.479686    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf"
Sep 12 00:51:06.479893 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:06.479703    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf" event={"ID":"d110ac11-1d50-4d03-963f-f8f93f722081","Type":"ContainerDied","Data":"56f6f07b9b7d3111a4053f33d24c2e179e3760fc189808a62bdbbcc316f850a9"}
Sep 12 00:51:06.479893 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:06.479762    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf" event={"ID":"d110ac11-1d50-4d03-963f-f8f93f722081","Type":"ContainerDied","Data":"19739555a920fc374d53b3af570b581a23c8752037b61d68e8927e2e55256bcb"}
Sep 12 00:51:06.479893 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:06.479796    2558 scope.go:117] "RemoveContainer" containerID="56f6f07b9b7d3111a4053f33d24c2e179e3760fc189808a62bdbbcc316f850a9"
Sep 12 00:51:06.490268 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:06.490245    2558 scope.go:117] "RemoveContainer" containerID="56f6f07b9b7d3111a4053f33d24c2e179e3760fc189808a62bdbbcc316f850a9"
Sep 12 00:51:06.490579 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:51:06.490553    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56f6f07b9b7d3111a4053f33d24c2e179e3760fc189808a62bdbbcc316f850a9\": container with ID starting with 56f6f07b9b7d3111a4053f33d24c2e179e3760fc189808a62bdbbcc316f850a9 not found: ID does not exist" containerID="56f6f07b9b7d3111a4053f33d24c2e179e3760fc189808a62bdbbcc316f850a9"
Sep 12 00:51:06.490625 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:06.490592    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56f6f07b9b7d3111a4053f33d24c2e179e3760fc189808a62bdbbcc316f850a9"} err="failed to get container status \"56f6f07b9b7d3111a4053f33d24c2e179e3760fc189808a62bdbbcc316f850a9\": rpc error: code = NotFound desc = could not find container \"56f6f07b9b7d3111a4053f33d24c2e179e3760fc189808a62bdbbcc316f850a9\": container with ID starting with 56f6f07b9b7d3111a4053f33d24c2e179e3760fc189808a62bdbbcc316f850a9 not found: ID does not exist"
Sep 12 00:51:06.540614 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:06.540585    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d110ac11-1d50-4d03-963f-f8f93f722081-metrics-certs\") pod \"d110ac11-1d50-4d03-963f-f8f93f722081\" (UID: \"d110ac11-1d50-4d03-963f-f8f93f722081\") "
Sep 12 00:51:06.540795 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:06.540660    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mnt5f\" (UniqueName: \"kubernetes.io/projected/d110ac11-1d50-4d03-963f-f8f93f722081-kube-api-access-mnt5f\") pod \"d110ac11-1d50-4d03-963f-f8f93f722081\" (UID: \"d110ac11-1d50-4d03-963f-f8f93f722081\") "
Sep 12 00:51:06.540795 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:06.540729    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d110ac11-1d50-4d03-963f-f8f93f722081-maas-api-tls\") pod \"d110ac11-1d50-4d03-963f-f8f93f722081\" (UID: \"d110ac11-1d50-4d03-963f-f8f93f722081\") "
Sep 12 00:51:06.540930 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:06.540821    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d110ac11-1d50-4d03-963f-f8f93f722081-openshift-service-ca\") pod \"d110ac11-1d50-4d03-963f-f8f93f722081\" (UID: \"d110ac11-1d50-4d03-963f-f8f93f722081\") "
Sep 12 00:51:06.541297 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:06.541266    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d110ac11-1d50-4d03-963f-f8f93f722081-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d110ac11-1d50-4d03-963f-f8f93f722081" (UID: "d110ac11-1d50-4d03-963f-f8f93f722081"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:51:06.543026 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:06.543005    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d110ac11-1d50-4d03-963f-f8f93f722081-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d110ac11-1d50-4d03-963f-f8f93f722081" (UID: "d110ac11-1d50-4d03-963f-f8f93f722081"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:51:06.543140 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:06.543026    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d110ac11-1d50-4d03-963f-f8f93f722081-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d110ac11-1d50-4d03-963f-f8f93f722081" (UID: "d110ac11-1d50-4d03-963f-f8f93f722081"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:51:06.543140 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:06.543078    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d110ac11-1d50-4d03-963f-f8f93f722081-kube-api-access-mnt5f" (OuterVolumeSpecName: "kube-api-access-mnt5f") pod "d110ac11-1d50-4d03-963f-f8f93f722081" (UID: "d110ac11-1d50-4d03-963f-f8f93f722081"). InnerVolumeSpecName "kube-api-access-mnt5f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:51:06.580243 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:06.580209    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:06.642533 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:06.642449    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d110ac11-1d50-4d03-963f-f8f93f722081-openshift-service-ca\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:51:06.642533 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:06.642485    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d110ac11-1d50-4d03-963f-f8f93f722081-metrics-certs\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:51:06.642533 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:06.642498    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mnt5f\" (UniqueName: \"kubernetes.io/projected/d110ac11-1d50-4d03-963f-f8f93f722081-kube-api-access-mnt5f\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:51:06.642533 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:06.642507    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d110ac11-1d50-4d03-963f-f8f93f722081-maas-api-tls\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:51:06.734141 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:06.734112    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:06.816267 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:06.816229    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf"]
Sep 12 00:51:06.820553 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:06.820524    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ee23f934-6bbbb67df4-zw9gf"]
Sep 12 00:51:06.997823 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:06.997793    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:07.107844 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:07.107799    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:07.739327 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:07.739298    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d110ac11-1d50-4d03-963f-f8f93f722081" path="/var/lib/kubelet/pods/d110ac11-1d50-4d03-963f-f8f93f722081/volumes"
Sep 12 00:51:08.254661 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:08.254627    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:09.340006 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:09.339969    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:10.562854 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:10.562821    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:11.718459 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:11.718427    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:13.028290 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:13.028254    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:15.532186 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:15.532152    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:15.644714 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:15.644643    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:16.062496 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:16.062447    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:16.293602 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:16.293572    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:16.446672 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:16.446643    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:16.569736 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:16.569702    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:23.423375 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:23.423338    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:23.581079 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:23.581047    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:23.698885 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:23.698853    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:23.948449 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:23.948412    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:25.749402 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:25.749371    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:26.003581 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:26.003493    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:28.058401 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:28.058366    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:28.999676 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:28.999638    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf"]
Sep 12 00:51:29.043840 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:29.043810    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:29.973316 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:29.973283    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:31.713250 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:31.713215    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:33.488492 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:33.488445    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:34.839493 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:34.839462    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:34.987333 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:34.987303    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:35.251694 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:35.251659    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:36.014259 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:36.014222    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:36.666650 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:36.666616    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:36.873356 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:36.873326    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:37.094187 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:37.094155    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:37.273267 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:37.273238    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:37.414323 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:37.414231    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:37.796324 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:37.796292    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:39.262366 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:39.262287    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:39.920420 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:39.920388    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:40.306438 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:40.306406    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:41.400785 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:41.400742    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:42.861502 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:42.861457    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:43.100647 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:43.100612    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:44.024890 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:44.024852    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf" podUID="b2566be6-5ba9-4718-9b0b-947cb094ba90" containerName="main" containerID="cri-o://a255307ee1d6a0244bb4e9bfc95515ba0d988022c92a3c3aedc87ec554bc08da" gracePeriod=45
Sep 12 00:51:44.286970 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:44.286909    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf"
Sep 12 00:51:44.335954 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:44.335920    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7kgtm\" (UniqueName: \"kubernetes.io/projected/b2566be6-5ba9-4718-9b0b-947cb094ba90-kube-api-access-7kgtm\") pod \"b2566be6-5ba9-4718-9b0b-947cb094ba90\" (UID: \"b2566be6-5ba9-4718-9b0b-947cb094ba90\") "
Sep 12 00:51:44.335954 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:44.335953    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2566be6-5ba9-4718-9b0b-947cb094ba90-tls-certs\") pod \"b2566be6-5ba9-4718-9b0b-947cb094ba90\" (UID: \"b2566be6-5ba9-4718-9b0b-947cb094ba90\") "
Sep 12 00:51:44.336238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:44.336063    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2566be6-5ba9-4718-9b0b-947cb094ba90-model-cache\") pod \"b2566be6-5ba9-4718-9b0b-947cb094ba90\" (UID: \"b2566be6-5ba9-4718-9b0b-947cb094ba90\") "
Sep 12 00:51:44.336238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:44.336106    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2566be6-5ba9-4718-9b0b-947cb094ba90-dshm\") pod \"b2566be6-5ba9-4718-9b0b-947cb094ba90\" (UID: \"b2566be6-5ba9-4718-9b0b-947cb094ba90\") "
Sep 12 00:51:44.336238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:44.336140    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2566be6-5ba9-4718-9b0b-947cb094ba90-home\") pod \"b2566be6-5ba9-4718-9b0b-947cb094ba90\" (UID: \"b2566be6-5ba9-4718-9b0b-947cb094ba90\") "
Sep 12 00:51:44.336238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:44.336189    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2566be6-5ba9-4718-9b0b-947cb094ba90-tmp-dir\") pod \"b2566be6-5ba9-4718-9b0b-947cb094ba90\" (UID: \"b2566be6-5ba9-4718-9b0b-947cb094ba90\") "
Sep 12 00:51:44.336675 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:44.336573    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2566be6-5ba9-4718-9b0b-947cb094ba90-model-cache" (OuterVolumeSpecName: "model-cache") pod "b2566be6-5ba9-4718-9b0b-947cb094ba90" (UID: "b2566be6-5ba9-4718-9b0b-947cb094ba90"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:51:44.336675 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:44.336636    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2566be6-5ba9-4718-9b0b-947cb094ba90-home" (OuterVolumeSpecName: "home") pod "b2566be6-5ba9-4718-9b0b-947cb094ba90" (UID: "b2566be6-5ba9-4718-9b0b-947cb094ba90"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:51:44.336849 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:44.336728    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2566be6-5ba9-4718-9b0b-947cb094ba90-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b2566be6-5ba9-4718-9b0b-947cb094ba90" (UID: "b2566be6-5ba9-4718-9b0b-947cb094ba90"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:51:44.338807 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:44.338782    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2566be6-5ba9-4718-9b0b-947cb094ba90-dshm" (OuterVolumeSpecName: "dshm") pod "b2566be6-5ba9-4718-9b0b-947cb094ba90" (UID: "b2566be6-5ba9-4718-9b0b-947cb094ba90"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:51:44.338919 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:44.338857    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2566be6-5ba9-4718-9b0b-947cb094ba90-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b2566be6-5ba9-4718-9b0b-947cb094ba90" (UID: "b2566be6-5ba9-4718-9b0b-947cb094ba90"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:51:44.338964 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:44.338926    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2566be6-5ba9-4718-9b0b-947cb094ba90-kube-api-access-7kgtm" (OuterVolumeSpecName: "kube-api-access-7kgtm") pod "b2566be6-5ba9-4718-9b0b-947cb094ba90" (UID: "b2566be6-5ba9-4718-9b0b-947cb094ba90"). InnerVolumeSpecName "kube-api-access-7kgtm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:51:44.437598 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:44.437562    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2566be6-5ba9-4718-9b0b-947cb094ba90-home\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:51:44.437598 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:44.437587    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2566be6-5ba9-4718-9b0b-947cb094ba90-tmp-dir\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:51:44.437598 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:44.437597    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kgtm\" (UniqueName: \"kubernetes.io/projected/b2566be6-5ba9-4718-9b0b-947cb094ba90-kube-api-access-7kgtm\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:51:44.437598 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:44.437606    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2566be6-5ba9-4718-9b0b-947cb094ba90-tls-certs\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:51:44.437856 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:44.437615    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2566be6-5ba9-4718-9b0b-947cb094ba90-model-cache\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:51:44.437856 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:44.437622    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2566be6-5ba9-4718-9b0b-947cb094ba90-dshm\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:51:44.464585 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:44.464543    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:44.686383 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:44.686355    2558 generic.go:356] "Generic (PLEG): container finished" podID="b2566be6-5ba9-4718-9b0b-947cb094ba90" containerID="a255307ee1d6a0244bb4e9bfc95515ba0d988022c92a3c3aedc87ec554bc08da" exitCode=0
Sep 12 00:51:44.686534 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:44.686399    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf"
Sep 12 00:51:44.686534 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:44.686440    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf" event={"ID":"b2566be6-5ba9-4718-9b0b-947cb094ba90","Type":"ContainerDied","Data":"a255307ee1d6a0244bb4e9bfc95515ba0d988022c92a3c3aedc87ec554bc08da"}
Sep 12 00:51:44.686534 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:44.686487    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf" event={"ID":"b2566be6-5ba9-4718-9b0b-947cb094ba90","Type":"ContainerDied","Data":"efc1041287f9ea811977c760001f1fa0f9310528b97591e6193ee25561c8c6c3"}
Sep 12 00:51:44.686534 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:44.686510    2558 scope.go:117] "RemoveContainer" containerID="a255307ee1d6a0244bb4e9bfc95515ba0d988022c92a3c3aedc87ec554bc08da"
Sep 12 00:51:44.696181 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:44.696160    2558 scope.go:117] "RemoveContainer" containerID="a255307ee1d6a0244bb4e9bfc95515ba0d988022c92a3c3aedc87ec554bc08da"
Sep 12 00:51:44.696488 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:51:44.696469    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a255307ee1d6a0244bb4e9bfc95515ba0d988022c92a3c3aedc87ec554bc08da\": container with ID starting with a255307ee1d6a0244bb4e9bfc95515ba0d988022c92a3c3aedc87ec554bc08da not found: ID does not exist" containerID="a255307ee1d6a0244bb4e9bfc95515ba0d988022c92a3c3aedc87ec554bc08da"
Sep 12 00:51:44.696541 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:44.696496    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a255307ee1d6a0244bb4e9bfc95515ba0d988022c92a3c3aedc87ec554bc08da"} err="failed to get container status \"a255307ee1d6a0244bb4e9bfc95515ba0d988022c92a3c3aedc87ec554bc08da\": rpc error: code = NotFound desc = could not find container \"a255307ee1d6a0244bb4e9bfc95515ba0d988022c92a3c3aedc87ec554bc08da\": container with ID starting with a255307ee1d6a0244bb4e9bfc95515ba0d988022c92a3c3aedc87ec554bc08da not found: ID does not exist"
Sep 12 00:51:44.748970 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:44.748939    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf"]
Sep 12 00:51:44.760277 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:44.760251    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-c3f8d8/explicit-model-c3f8d8-kserve-84dc698d45-xc8xf"]
Sep 12 00:51:45.634245 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:45.634165    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:45.735570 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:45.735525    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:45.738564 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:45.738540    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2566be6-5ba9-4718-9b0b-947cb094ba90" path="/var/lib/kubelet/pods/b2566be6-5ba9-4718-9b0b-947cb094ba90/volumes"
Sep 12 00:51:45.853833 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:45.853793    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:46.197885 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:46.197849    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:46.303937 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:46.303904    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:47.005882 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:47.005848    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:47.883285 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:47.883249    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:48.720121 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:48.720059    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:49.744002 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:49.743971    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:49.990545 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:49.990511    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:50.421494 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:50.421450    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:50.766512 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:50.766482    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:51.083187 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:51.083107    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:52.021635 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:52.021572    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:52.191793 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:52.191749    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:52.293310 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:52.293229    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:51:56.267275 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:51:56.267235    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:02.234502 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:02.234461    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:07.098873 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:07.098840    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:08.341219 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:08.341185    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9"]
Sep 12 00:52:08.341629 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:08.341616    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2566be6-5ba9-4718-9b0b-947cb094ba90" containerName="main"
Sep 12 00:52:08.341675 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:08.341630    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2566be6-5ba9-4718-9b0b-947cb094ba90" containerName="main"
Sep 12 00:52:08.341675 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:08.341639    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d110ac11-1d50-4d03-963f-f8f93f722081" containerName="maas-api"
Sep 12 00:52:08.341675 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:08.341644    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d110ac11-1d50-4d03-963f-f8f93f722081" containerName="maas-api"
Sep 12 00:52:08.341768 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:08.341714    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2566be6-5ba9-4718-9b0b-947cb094ba90" containerName="main"
Sep 12 00:52:08.341768 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:08.341723    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d110ac11-1d50-4d03-963f-f8f93f722081" containerName="maas-api"
Sep 12 00:52:08.344541 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:08.344525    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9"
Sep 12 00:52:08.349381 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:08.349355    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-4d485e\"/\"openshift-service-ca.crt\""
Sep 12 00:52:08.350522 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:08.350499    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-4d485e\"/\"default-dockercfg-nvltd\""
Sep 12 00:52:08.350645 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:08.350534    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-4d485e\"/\"distinct-w0-4d485e-kserve-self-signed-certs\""
Sep 12 00:52:08.350645 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:08.350568    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-4d485e\"/\"kube-root-ca.crt\""
Sep 12 00:52:08.360770 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:08.360744    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9"]
Sep 12 00:52:08.456283 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:08.456246    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:08.498503 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:08.498462    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-home\") pod \"distinct-w0-4d485e-kserve-9b7f797d5-l5fg9\" (UID: \"78e29efa-f066-4fc5-a761-89dcf4ecb7a1\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9"
Sep 12 00:52:08.498690 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:08.498528    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kj9t7\" (UniqueName: \"kubernetes.io/projected/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-kube-api-access-kj9t7\") pod \"distinct-w0-4d485e-kserve-9b7f797d5-l5fg9\" (UID: \"78e29efa-f066-4fc5-a761-89dcf4ecb7a1\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9"
Sep 12 00:52:08.498690 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:08.498564    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-model-cache\") pod \"distinct-w0-4d485e-kserve-9b7f797d5-l5fg9\" (UID: \"78e29efa-f066-4fc5-a761-89dcf4ecb7a1\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9"
Sep 12 00:52:08.498690 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:08.498622    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-tls-certs\") pod \"distinct-w0-4d485e-kserve-9b7f797d5-l5fg9\" (UID: \"78e29efa-f066-4fc5-a761-89dcf4ecb7a1\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9"
Sep 12 00:52:08.498690 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:08.498650    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-dshm\") pod \"distinct-w0-4d485e-kserve-9b7f797d5-l5fg9\" (UID: \"78e29efa-f066-4fc5-a761-89dcf4ecb7a1\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9"
Sep 12 00:52:08.498690 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:08.498686    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-tmp-dir\") pod \"distinct-w0-4d485e-kserve-9b7f797d5-l5fg9\" (UID: \"78e29efa-f066-4fc5-a761-89dcf4ecb7a1\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9"
Sep 12 00:52:08.600065 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:08.599967    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-home\") pod \"distinct-w0-4d485e-kserve-9b7f797d5-l5fg9\" (UID: \"78e29efa-f066-4fc5-a761-89dcf4ecb7a1\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9"
Sep 12 00:52:08.600065 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:08.600020    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kj9t7\" (UniqueName: \"kubernetes.io/projected/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-kube-api-access-kj9t7\") pod \"distinct-w0-4d485e-kserve-9b7f797d5-l5fg9\" (UID: \"78e29efa-f066-4fc5-a761-89dcf4ecb7a1\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9"
Sep 12 00:52:08.600065 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:08.600056    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-model-cache\") pod \"distinct-w0-4d485e-kserve-9b7f797d5-l5fg9\" (UID: \"78e29efa-f066-4fc5-a761-89dcf4ecb7a1\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9"
Sep 12 00:52:08.600394 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:08.600208    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-tls-certs\") pod \"distinct-w0-4d485e-kserve-9b7f797d5-l5fg9\" (UID: \"78e29efa-f066-4fc5-a761-89dcf4ecb7a1\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9"
Sep 12 00:52:08.600394 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:08.600252    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-dshm\") pod \"distinct-w0-4d485e-kserve-9b7f797d5-l5fg9\" (UID: \"78e29efa-f066-4fc5-a761-89dcf4ecb7a1\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9"
Sep 12 00:52:08.600394 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:08.600296    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-tmp-dir\") pod \"distinct-w0-4d485e-kserve-9b7f797d5-l5fg9\" (UID: \"78e29efa-f066-4fc5-a761-89dcf4ecb7a1\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9"
Sep 12 00:52:08.600550 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:08.600491    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-home\") pod \"distinct-w0-4d485e-kserve-9b7f797d5-l5fg9\" (UID: \"78e29efa-f066-4fc5-a761-89dcf4ecb7a1\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9"
Sep 12 00:52:08.600550 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:08.600541    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-model-cache\") pod \"distinct-w0-4d485e-kserve-9b7f797d5-l5fg9\" (UID: \"78e29efa-f066-4fc5-a761-89dcf4ecb7a1\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9"
Sep 12 00:52:08.600619 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:08.600603    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-tmp-dir\") pod \"distinct-w0-4d485e-kserve-9b7f797d5-l5fg9\" (UID: \"78e29efa-f066-4fc5-a761-89dcf4ecb7a1\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9"
Sep 12 00:52:08.602787 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:08.602762    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-dshm\") pod \"distinct-w0-4d485e-kserve-9b7f797d5-l5fg9\" (UID: \"78e29efa-f066-4fc5-a761-89dcf4ecb7a1\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9"
Sep 12 00:52:08.602959 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:08.602943    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-tls-certs\") pod \"distinct-w0-4d485e-kserve-9b7f797d5-l5fg9\" (UID: \"78e29efa-f066-4fc5-a761-89dcf4ecb7a1\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9"
Sep 12 00:52:08.615235 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:08.615208    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kj9t7\" (UniqueName: \"kubernetes.io/projected/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-kube-api-access-kj9t7\") pod \"distinct-w0-4d485e-kserve-9b7f797d5-l5fg9\" (UID: \"78e29efa-f066-4fc5-a761-89dcf4ecb7a1\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9"
Sep 12 00:52:08.654186 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:08.654145    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9"
Sep 12 00:52:08.954332 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:08.954302    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9"]
Sep 12 00:52:08.955425 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:52:08.955399    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod78e29efa_f066_4fc5_a761_89dcf4ecb7a1.slice/crio-f23fa8a135b6977e0ecd690c580a3a026dbcce9e6c2d21973933a781a69824e5 WatchSource:0}: Error finding container f23fa8a135b6977e0ecd690c580a3a026dbcce9e6c2d21973933a781a69824e5: Status 404 returned error can't find the container with id f23fa8a135b6977e0ecd690c580a3a026dbcce9e6c2d21973933a781a69824e5
Sep 12 00:52:09.046588 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:09.046553    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:09.694561 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:09.694525    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:09.820398 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:09.820356    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9" event={"ID":"78e29efa-f066-4fc5-a761-89dcf4ecb7a1","Type":"ContainerStarted","Data":"fff6499613415564ec7b885e9d5e0ad9e3bebb18628c4902f8d9ea503991ab27"}
Sep 12 00:52:09.820566 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:09.820407    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9" event={"ID":"78e29efa-f066-4fc5-a761-89dcf4ecb7a1","Type":"ContainerStarted","Data":"f23fa8a135b6977e0ecd690c580a3a026dbcce9e6c2d21973933a781a69824e5"}
Sep 12 00:52:09.852563 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:09.852511    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9" podStartSLOduration=1.852495324 podStartE2EDuration="1.852495324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:52:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:52:09.850373178 +0000 UTC m=+1366.524432038" watchObservedRunningTime="2026-09-12 00:52:09.852495324 +0000 UTC m=+1366.526554184"
Sep 12 00:52:10.051290 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:10.051184    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:10.168269 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:10.168231    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:10.380340 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:10.380243    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:10.864328 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:10.864278    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:11.263734 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:11.263689    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:12.326759 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:12.326712    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:13.365112 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:13.365058    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:13.881472 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:13.881441    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:13.978953 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:13.978920    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:14.074586 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:14.074547    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5"]
Sep 12 00:52:14.098310 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:14.098279    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:14.347917 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:14.347886    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:15.372037 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:15.371995    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:15.470704 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:15.470677    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:15.575521 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:15.575469    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:15.966276 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:15.966248    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:16.077598 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:16.077558    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:16.402599 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:16.402510    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:17.606207 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:17.606161    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:18.391374 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:18.391344    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:18.654817 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:18.654727    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9"
Sep 12 00:52:18.654817 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:18.654810    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9"
Sep 12 00:52:18.667755 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:18.667733    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9"
Sep 12 00:52:18.884797 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:18.884763    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9"
Sep 12 00:52:19.278423 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:19.278389    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:20.008027 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:20.007996    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:29.095413 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:29.095363    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5" podUID="4758dcdb-0efd-4893-a54d-b5af9deff917" containerName="main" containerID="cri-o://7e39bc597571332751a3bcae70eac74d8b2f244c0f0b3d1f8b9f01b4cb91f73d" gracePeriod=45
Sep 12 00:52:29.351587 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:29.351519    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5"
Sep 12 00:52:29.412443 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:29.412415    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4758dcdb-0efd-4893-a54d-b5af9deff917-tmp-dir\") pod \"4758dcdb-0efd-4893-a54d-b5af9deff917\" (UID: \"4758dcdb-0efd-4893-a54d-b5af9deff917\") "
Sep 12 00:52:29.412443 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:29.412445    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4758dcdb-0efd-4893-a54d-b5af9deff917-model-cache\") pod \"4758dcdb-0efd-4893-a54d-b5af9deff917\" (UID: \"4758dcdb-0efd-4893-a54d-b5af9deff917\") "
Sep 12 00:52:29.412695 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:29.412500    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4758dcdb-0efd-4893-a54d-b5af9deff917-dshm\") pod \"4758dcdb-0efd-4893-a54d-b5af9deff917\" (UID: \"4758dcdb-0efd-4893-a54d-b5af9deff917\") "
Sep 12 00:52:29.412695 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:29.412533    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4758dcdb-0efd-4893-a54d-b5af9deff917-tls-certs\") pod \"4758dcdb-0efd-4893-a54d-b5af9deff917\" (UID: \"4758dcdb-0efd-4893-a54d-b5af9deff917\") "
Sep 12 00:52:29.412695 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:29.412580    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4md7\" (UniqueName: \"kubernetes.io/projected/4758dcdb-0efd-4893-a54d-b5af9deff917-kube-api-access-l4md7\") pod \"4758dcdb-0efd-4893-a54d-b5af9deff917\" (UID: \"4758dcdb-0efd-4893-a54d-b5af9deff917\") "
Sep 12 00:52:29.412695 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:29.412638    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4758dcdb-0efd-4893-a54d-b5af9deff917-home\") pod \"4758dcdb-0efd-4893-a54d-b5af9deff917\" (UID: \"4758dcdb-0efd-4893-a54d-b5af9deff917\") "
Sep 12 00:52:29.412916 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:29.412734    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4758dcdb-0efd-4893-a54d-b5af9deff917-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4758dcdb-0efd-4893-a54d-b5af9deff917" (UID: "4758dcdb-0efd-4893-a54d-b5af9deff917"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:52:29.412916 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:29.412822    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4758dcdb-0efd-4893-a54d-b5af9deff917-model-cache" (OuterVolumeSpecName: "model-cache") pod "4758dcdb-0efd-4893-a54d-b5af9deff917" (UID: "4758dcdb-0efd-4893-a54d-b5af9deff917"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:52:29.413121 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:29.413079    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4758dcdb-0efd-4893-a54d-b5af9deff917-tmp-dir\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:52:29.413199 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:29.413125    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4758dcdb-0efd-4893-a54d-b5af9deff917-model-cache\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:52:29.413199 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:29.413122    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4758dcdb-0efd-4893-a54d-b5af9deff917-home" (OuterVolumeSpecName: "home") pod "4758dcdb-0efd-4893-a54d-b5af9deff917" (UID: "4758dcdb-0efd-4893-a54d-b5af9deff917"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:52:29.414940 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:29.414915    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4758dcdb-0efd-4893-a54d-b5af9deff917-dshm" (OuterVolumeSpecName: "dshm") pod "4758dcdb-0efd-4893-a54d-b5af9deff917" (UID: "4758dcdb-0efd-4893-a54d-b5af9deff917"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:52:29.415044 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:29.414978    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4758dcdb-0efd-4893-a54d-b5af9deff917-kube-api-access-l4md7" (OuterVolumeSpecName: "kube-api-access-l4md7") pod "4758dcdb-0efd-4893-a54d-b5af9deff917" (UID: "4758dcdb-0efd-4893-a54d-b5af9deff917"). InnerVolumeSpecName "kube-api-access-l4md7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:52:29.415044 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:29.414985    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4758dcdb-0efd-4893-a54d-b5af9deff917-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4758dcdb-0efd-4893-a54d-b5af9deff917" (UID: "4758dcdb-0efd-4893-a54d-b5af9deff917"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:52:29.514487 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:29.514450    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4758dcdb-0efd-4893-a54d-b5af9deff917-dshm\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:52:29.514487 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:29.514480    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4758dcdb-0efd-4893-a54d-b5af9deff917-tls-certs\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:52:29.514487 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:29.514492    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4md7\" (UniqueName: \"kubernetes.io/projected/4758dcdb-0efd-4893-a54d-b5af9deff917-kube-api-access-l4md7\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:52:29.514765 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:29.514501    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4758dcdb-0efd-4893-a54d-b5af9deff917-home\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:52:29.934980 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:29.934942    2558 generic.go:356] "Generic (PLEG): container finished" podID="4758dcdb-0efd-4893-a54d-b5af9deff917" containerID="7e39bc597571332751a3bcae70eac74d8b2f244c0f0b3d1f8b9f01b4cb91f73d" exitCode=0
Sep 12 00:52:29.934980 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:29.934985    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5" event={"ID":"4758dcdb-0efd-4893-a54d-b5af9deff917","Type":"ContainerDied","Data":"7e39bc597571332751a3bcae70eac74d8b2f244c0f0b3d1f8b9f01b4cb91f73d"}
Sep 12 00:52:29.935239 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:29.934999    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5"
Sep 12 00:52:29.935239 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:29.935020    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5" event={"ID":"4758dcdb-0efd-4893-a54d-b5af9deff917","Type":"ContainerDied","Data":"39035228fc906c75798f1a8e7d7c2e2e5780bc158ff7a9f923e9818b578bf8cf"}
Sep 12 00:52:29.935239 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:29.935038    2558 scope.go:117] "RemoveContainer" containerID="7e39bc597571332751a3bcae70eac74d8b2f244c0f0b3d1f8b9f01b4cb91f73d"
Sep 12 00:52:29.944205 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:29.944185    2558 scope.go:117] "RemoveContainer" containerID="7e39bc597571332751a3bcae70eac74d8b2f244c0f0b3d1f8b9f01b4cb91f73d"
Sep 12 00:52:29.944471 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:52:29.944451    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e39bc597571332751a3bcae70eac74d8b2f244c0f0b3d1f8b9f01b4cb91f73d\": container with ID starting with 7e39bc597571332751a3bcae70eac74d8b2f244c0f0b3d1f8b9f01b4cb91f73d not found: ID does not exist" containerID="7e39bc597571332751a3bcae70eac74d8b2f244c0f0b3d1f8b9f01b4cb91f73d"
Sep 12 00:52:29.944530 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:29.944477    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e39bc597571332751a3bcae70eac74d8b2f244c0f0b3d1f8b9f01b4cb91f73d"} err="failed to get container status \"7e39bc597571332751a3bcae70eac74d8b2f244c0f0b3d1f8b9f01b4cb91f73d\": rpc error: code = NotFound desc = could not find container \"7e39bc597571332751a3bcae70eac74d8b2f244c0f0b3d1f8b9f01b4cb91f73d\": container with ID starting with 7e39bc597571332751a3bcae70eac74d8b2f244c0f0b3d1f8b9f01b4cb91f73d not found: ID does not exist"
Sep 12 00:52:29.953043 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:29.953020    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5"]
Sep 12 00:52:29.957423 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:29.957402    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-81df64/test-model-81df64-kserve-65847c6f5-tcjt5"]
Sep 12 00:52:31.738148 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:31.738108    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4758dcdb-0efd-4893-a54d-b5af9deff917" path="/var/lib/kubelet/pods/4758dcdb-0efd-4893-a54d-b5af9deff917/volumes"
Sep 12 00:52:34.816283 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:34.816241    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4"]
Sep 12 00:52:34.816812 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:34.816729    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4758dcdb-0efd-4893-a54d-b5af9deff917" containerName="main"
Sep 12 00:52:34.816812 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:34.816742    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4758dcdb-0efd-4893-a54d-b5af9deff917" containerName="main"
Sep 12 00:52:34.816928 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:34.816846    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="4758dcdb-0efd-4893-a54d-b5af9deff917" containerName="main"
Sep 12 00:52:34.821342 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:34.821312    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4"
Sep 12 00:52:34.825734 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:34.825704    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-904cd0\"/\"default-dockercfg-c79c7\""
Sep 12 00:52:34.825931 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:34.825705    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-904cd0\"/\"kube-root-ca.crt\""
Sep 12 00:52:34.825931 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:34.825705    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-904cd0\"/\"openshift-service-ca.crt\""
Sep 12 00:52:34.825931 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:34.825781    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-904cd0\"/\"noten-model-904cd0-kserve-self-signed-certs\""
Sep 12 00:52:34.836645 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:34.836616    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4"]
Sep 12 00:52:34.971387 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:34.971349    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0eb8255b-f9fa-4570-a417-17c1845bcffc-tmp-dir\") pod \"noten-model-904cd0-kserve-9fb476dcf-q7zl4\" (UID: \"0eb8255b-f9fa-4570-a417-17c1845bcffc\") " pod="ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4"
Sep 12 00:52:34.971582 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:34.971406    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0eb8255b-f9fa-4570-a417-17c1845bcffc-dshm\") pod \"noten-model-904cd0-kserve-9fb476dcf-q7zl4\" (UID: \"0eb8255b-f9fa-4570-a417-17c1845bcffc\") " pod="ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4"
Sep 12 00:52:34.971582 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:34.971534    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0eb8255b-f9fa-4570-a417-17c1845bcffc-home\") pod \"noten-model-904cd0-kserve-9fb476dcf-q7zl4\" (UID: \"0eb8255b-f9fa-4570-a417-17c1845bcffc\") " pod="ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4"
Sep 12 00:52:34.971664 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:34.971578    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0eb8255b-f9fa-4570-a417-17c1845bcffc-tls-certs\") pod \"noten-model-904cd0-kserve-9fb476dcf-q7zl4\" (UID: \"0eb8255b-f9fa-4570-a417-17c1845bcffc\") " pod="ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4"
Sep 12 00:52:34.971664 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:34.971647    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0eb8255b-f9fa-4570-a417-17c1845bcffc-model-cache\") pod \"noten-model-904cd0-kserve-9fb476dcf-q7zl4\" (UID: \"0eb8255b-f9fa-4570-a417-17c1845bcffc\") " pod="ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4"
Sep 12 00:52:34.971764 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:34.971672    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffr8r\" (UniqueName: \"kubernetes.io/projected/0eb8255b-f9fa-4570-a417-17c1845bcffc-kube-api-access-ffr8r\") pod \"noten-model-904cd0-kserve-9fb476dcf-q7zl4\" (UID: \"0eb8255b-f9fa-4570-a417-17c1845bcffc\") " pod="ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4"
Sep 12 00:52:35.072510 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:35.072432    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0eb8255b-f9fa-4570-a417-17c1845bcffc-model-cache\") pod \"noten-model-904cd0-kserve-9fb476dcf-q7zl4\" (UID: \"0eb8255b-f9fa-4570-a417-17c1845bcffc\") " pod="ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4"
Sep 12 00:52:35.072510 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:35.072483    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffr8r\" (UniqueName: \"kubernetes.io/projected/0eb8255b-f9fa-4570-a417-17c1845bcffc-kube-api-access-ffr8r\") pod \"noten-model-904cd0-kserve-9fb476dcf-q7zl4\" (UID: \"0eb8255b-f9fa-4570-a417-17c1845bcffc\") " pod="ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4"
Sep 12 00:52:35.072736 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:35.072521    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0eb8255b-f9fa-4570-a417-17c1845bcffc-tmp-dir\") pod \"noten-model-904cd0-kserve-9fb476dcf-q7zl4\" (UID: \"0eb8255b-f9fa-4570-a417-17c1845bcffc\") " pod="ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4"
Sep 12 00:52:35.072736 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:35.072568    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0eb8255b-f9fa-4570-a417-17c1845bcffc-dshm\") pod \"noten-model-904cd0-kserve-9fb476dcf-q7zl4\" (UID: \"0eb8255b-f9fa-4570-a417-17c1845bcffc\") " pod="ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4"
Sep 12 00:52:35.072736 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:35.072656    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0eb8255b-f9fa-4570-a417-17c1845bcffc-home\") pod \"noten-model-904cd0-kserve-9fb476dcf-q7zl4\" (UID: \"0eb8255b-f9fa-4570-a417-17c1845bcffc\") " pod="ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4"
Sep 12 00:52:35.072736 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:35.072699    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0eb8255b-f9fa-4570-a417-17c1845bcffc-tls-certs\") pod \"noten-model-904cd0-kserve-9fb476dcf-q7zl4\" (UID: \"0eb8255b-f9fa-4570-a417-17c1845bcffc\") " pod="ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4"
Sep 12 00:52:35.072963 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:35.072896    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0eb8255b-f9fa-4570-a417-17c1845bcffc-model-cache\") pod \"noten-model-904cd0-kserve-9fb476dcf-q7zl4\" (UID: \"0eb8255b-f9fa-4570-a417-17c1845bcffc\") " pod="ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4"
Sep 12 00:52:35.072963 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:35.072911    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0eb8255b-f9fa-4570-a417-17c1845bcffc-tmp-dir\") pod \"noten-model-904cd0-kserve-9fb476dcf-q7zl4\" (UID: \"0eb8255b-f9fa-4570-a417-17c1845bcffc\") " pod="ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4"
Sep 12 00:52:35.073098 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:35.073048    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0eb8255b-f9fa-4570-a417-17c1845bcffc-home\") pod \"noten-model-904cd0-kserve-9fb476dcf-q7zl4\" (UID: \"0eb8255b-f9fa-4570-a417-17c1845bcffc\") " pod="ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4"
Sep 12 00:52:35.075325 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:35.075299    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0eb8255b-f9fa-4570-a417-17c1845bcffc-dshm\") pod \"noten-model-904cd0-kserve-9fb476dcf-q7zl4\" (UID: \"0eb8255b-f9fa-4570-a417-17c1845bcffc\") " pod="ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4"
Sep 12 00:52:35.075614 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:35.075595    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0eb8255b-f9fa-4570-a417-17c1845bcffc-tls-certs\") pod \"noten-model-904cd0-kserve-9fb476dcf-q7zl4\" (UID: \"0eb8255b-f9fa-4570-a417-17c1845bcffc\") " pod="ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4"
Sep 12 00:52:35.083014 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:35.082987    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffr8r\" (UniqueName: \"kubernetes.io/projected/0eb8255b-f9fa-4570-a417-17c1845bcffc-kube-api-access-ffr8r\") pod \"noten-model-904cd0-kserve-9fb476dcf-q7zl4\" (UID: \"0eb8255b-f9fa-4570-a417-17c1845bcffc\") " pod="ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4"
Sep 12 00:52:35.115894 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:35.115864    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:35.132890 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:35.132847    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4"
Sep 12 00:52:35.407646 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:35.407624    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4"]
Sep 12 00:52:35.409388 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:52:35.409353    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0eb8255b_f9fa_4570_a417_17c1845bcffc.slice/crio-099932bac337a5b1d4c083a464a3c94cc910bb418da7ba43c5eed7171a550996 WatchSource:0}: Error finding container 099932bac337a5b1d4c083a464a3c94cc910bb418da7ba43c5eed7171a550996: Status 404 returned error can't find the container with id 099932bac337a5b1d4c083a464a3c94cc910bb418da7ba43c5eed7171a550996
Sep 12 00:52:35.966980 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:35.966943    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4" event={"ID":"0eb8255b-f9fa-4570-a417-17c1845bcffc","Type":"ContainerStarted","Data":"adc75f1c8f7d5aa6cbd9db50e9dbdf180cde0bc6a9e246fc75b2a61a93fc6543"}
Sep 12 00:52:35.967411 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:35.966988    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4" event={"ID":"0eb8255b-f9fa-4570-a417-17c1845bcffc","Type":"ContainerStarted","Data":"099932bac337a5b1d4c083a464a3c94cc910bb418da7ba43c5eed7171a550996"}
Sep 12 00:52:35.989251 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:35.989199    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4" podStartSLOduration=1.989183967 podStartE2EDuration="1.989183967s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:52:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:52:35.988317846 +0000 UTC m=+1392.662376734" watchObservedRunningTime="2026-09-12 00:52:35.989183967 +0000 UTC m=+1392.663242880"
Sep 12 00:52:37.587772 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:37.587698    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:38.268441 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:38.268403    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:38.680467 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:38.680382    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:39.077059 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:39.077030    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:39.409131 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:39.409033    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7676-640920"]
Sep 12 00:52:39.412548 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:39.412526    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7676-640920"
Sep 12 00:52:39.415207 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:39.415185    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tqwft\""
Sep 12 00:52:39.417414 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:39.417388    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7676-640920"]
Sep 12 00:52:39.519186 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:39.519154    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8xdh\" (UniqueName: \"kubernetes.io/projected/6fd2725a-39d1-42bd-bbc4-da57f052e676-kube-api-access-p8xdh\") pod \"test-curl-7676-640920\" (UID: \"6fd2725a-39d1-42bd-bbc4-da57f052e676\") " pod="opendatahub/test-curl-7676-640920"
Sep 12 00:52:39.620791 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:39.620753    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8xdh\" (UniqueName: \"kubernetes.io/projected/6fd2725a-39d1-42bd-bbc4-da57f052e676-kube-api-access-p8xdh\") pod \"test-curl-7676-640920\" (UID: \"6fd2725a-39d1-42bd-bbc4-da57f052e676\") " pod="opendatahub/test-curl-7676-640920"
Sep 12 00:52:39.628631 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:39.628602    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8xdh\" (UniqueName: \"kubernetes.io/projected/6fd2725a-39d1-42bd-bbc4-da57f052e676-kube-api-access-p8xdh\") pod \"test-curl-7676-640920\" (UID: \"6fd2725a-39d1-42bd-bbc4-da57f052e676\") " pod="opendatahub/test-curl-7676-640920"
Sep 12 00:52:39.724820 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:39.724790    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7676-640920"
Sep 12 00:52:39.782996 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:39.782962    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:40.004685 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:40.004662    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7676-640920"]
Sep 12 00:52:40.006342 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:52:40.006314    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6fd2725a_39d1_42bd_bbc4_da57f052e676.slice/crio-b497d41b0395f98c2b88c682e65e69c2d39ebc393ee949e2a0ba9c5312dc8e98 WatchSource:0}: Error finding container b497d41b0395f98c2b88c682e65e69c2d39ebc393ee949e2a0ba9c5312dc8e98: Status 404 returned error can't find the container with id b497d41b0395f98c2b88c682e65e69c2d39ebc393ee949e2a0ba9c5312dc8e98
Sep 12 00:52:40.565226 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:40.565183    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:40.994332 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:40.994294    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7676-640920" event={"ID":"6fd2725a-39d1-42bd-bbc4-da57f052e676","Type":"ContainerStarted","Data":"fb1222aca7a690a4e7e7f394d2ac4ea3fc9b0fe08ce15d7705cf091bda52889c"}
Sep 12 00:52:40.994782 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:40.994341    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7676-640920" event={"ID":"6fd2725a-39d1-42bd-bbc4-da57f052e676","Type":"ContainerStarted","Data":"b497d41b0395f98c2b88c682e65e69c2d39ebc393ee949e2a0ba9c5312dc8e98"}
Sep 12 00:52:41.009661 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:41.009606    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7676-640920" podStartSLOduration=2.009589699 podStartE2EDuration="2.009589699s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:52:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:52:41.007757154 +0000 UTC m=+1397.681816013" watchObservedRunningTime="2026-09-12 00:52:41.009589699 +0000 UTC m=+1397.683648557"
Sep 12 00:52:41.277298 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:41.277194    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:41.591848 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:41.591755    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7676-640920"]
Sep 12 00:52:41.594325 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:41.594298    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7676-640920"]
Sep 12 00:52:41.703895 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:41.703868    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7676-a482ff"]
Sep 12 00:52:41.704408 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:41.704392    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fd2725a-39d1-42bd-bbc4-da57f052e676" containerName="test-curl-7676-640920"
Sep 12 00:52:41.704466 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:41.704413    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fd2725a-39d1-42bd-bbc4-da57f052e676" containerName="test-curl-7676-640920"
Sep 12 00:52:41.704504 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:41.704482    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fd2725a-39d1-42bd-bbc4-da57f052e676" containerName="test-curl-7676-640920"
Sep 12 00:52:41.707722 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:41.707705    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7676-a482ff"
Sep 12 00:52:41.713664 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:41.713640    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7676-a482ff"]
Sep 12 00:52:41.843854 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:41.843750    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-st7xz\" (UniqueName: \"kubernetes.io/projected/d57e57e6-4ce3-4579-bc3d-8fba02271e14-kube-api-access-st7xz\") pod \"test-curl-7676-a482ff\" (UID: \"d57e57e6-4ce3-4579-bc3d-8fba02271e14\") " pod="opendatahub/test-curl-7676-a482ff"
Sep 12 00:52:41.945208 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:41.945171    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-st7xz\" (UniqueName: \"kubernetes.io/projected/d57e57e6-4ce3-4579-bc3d-8fba02271e14-kube-api-access-st7xz\") pod \"test-curl-7676-a482ff\" (UID: \"d57e57e6-4ce3-4579-bc3d-8fba02271e14\") " pod="opendatahub/test-curl-7676-a482ff"
Sep 12 00:52:41.955914 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:41.955881    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-st7xz\" (UniqueName: \"kubernetes.io/projected/d57e57e6-4ce3-4579-bc3d-8fba02271e14-kube-api-access-st7xz\") pod \"test-curl-7676-a482ff\" (UID: \"d57e57e6-4ce3-4579-bc3d-8fba02271e14\") " pod="opendatahub/test-curl-7676-a482ff"
Sep 12 00:52:41.963920 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:41.963871    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:42.000191 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:52:42.000159    2558 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7676-640920" container="test-curl-7676-640920"
Sep 12 00:52:42.019116 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:42.019060    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7676-a482ff"
Sep 12 00:52:42.289025 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:42.289004    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7676-a482ff"]
Sep 12 00:52:42.291024 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:52:42.291001    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd57e57e6_4ce3_4579_bc3d_8fba02271e14.slice/crio-b47321467f50f46a4b2af78c5d545da03d785d1fb9755d0e01d828073ef14013 WatchSource:0}: Error finding container b47321467f50f46a4b2af78c5d545da03d785d1fb9755d0e01d828073ef14013: Status 404 returned error can't find the container with id b47321467f50f46a4b2af78c5d545da03d785d1fb9755d0e01d828073ef14013
Sep 12 00:52:43.006453 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:43.006415    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7676-a482ff" event={"ID":"d57e57e6-4ce3-4579-bc3d-8fba02271e14","Type":"ContainerStarted","Data":"748c303f947cad6c43a04cb6288407bdd8c8aa95a7ad04a2d2ba223c6bf19325"}
Sep 12 00:52:43.006942 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:43.006463    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7676-a482ff" event={"ID":"d57e57e6-4ce3-4579-bc3d-8fba02271e14","Type":"ContainerStarted","Data":"b47321467f50f46a4b2af78c5d545da03d785d1fb9755d0e01d828073ef14013"}
Sep 12 00:52:43.006942 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:43.006587    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7676-640920" podUID="6fd2725a-39d1-42bd-bbc4-da57f052e676" containerName="test-curl-7676-640920" containerID="cri-o://fb1222aca7a690a4e7e7f394d2ac4ea3fc9b0fe08ce15d7705cf091bda52889c" gracePeriod=2
Sep 12 00:52:43.022066 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:43.022013    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7676-a482ff" podStartSLOduration=2.021996707 podStartE2EDuration="2.021996707s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:52:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:52:43.020428466 +0000 UTC m=+1399.694487326" watchObservedRunningTime="2026-09-12 00:52:43.021996707 +0000 UTC m=+1399.696055609"
Sep 12 00:52:43.601098 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:43.601053    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7676-a482ff"]
Sep 12 00:52:43.605642 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:43.605613    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7676-a482ff"]
Sep 12 00:52:43.711233 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:43.711203    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7676-7cd785"]
Sep 12 00:52:43.711674 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:43.711659    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d57e57e6-4ce3-4579-bc3d-8fba02271e14" containerName="test-curl-7676-a482ff"
Sep 12 00:52:43.711737 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:43.711677    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d57e57e6-4ce3-4579-bc3d-8fba02271e14" containerName="test-curl-7676-a482ff"
Sep 12 00:52:43.711779 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:43.711751    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d57e57e6-4ce3-4579-bc3d-8fba02271e14" containerName="test-curl-7676-a482ff"
Sep 12 00:52:43.714935 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:43.714914    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7676-7cd785"
Sep 12 00:52:43.718604 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:43.718582    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7676-7cd785"]
Sep 12 00:52:43.863698 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:43.863668    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqqq8\" (UniqueName: \"kubernetes.io/projected/064d5746-54f3-43c1-a138-a20f7f41d72c-kube-api-access-mqqq8\") pod \"test-curl-7676-7cd785\" (UID: \"064d5746-54f3-43c1-a138-a20f7f41d72c\") " pod="opendatahub/test-curl-7676-7cd785"
Sep 12 00:52:43.964927 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:43.964848    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqqq8\" (UniqueName: \"kubernetes.io/projected/064d5746-54f3-43c1-a138-a20f7f41d72c-kube-api-access-mqqq8\") pod \"test-curl-7676-7cd785\" (UID: \"064d5746-54f3-43c1-a138-a20f7f41d72c\") " pod="opendatahub/test-curl-7676-7cd785"
Sep 12 00:52:43.973719 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:43.973688    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqqq8\" (UniqueName: \"kubernetes.io/projected/064d5746-54f3-43c1-a138-a20f7f41d72c-kube-api-access-mqqq8\") pod \"test-curl-7676-7cd785\" (UID: \"064d5746-54f3-43c1-a138-a20f7f41d72c\") " pod="opendatahub/test-curl-7676-7cd785"
Sep 12 00:52:44.011329 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:52:44.011302    2558 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7676-a482ff" container="test-curl-7676-a482ff"
Sep 12 00:52:44.026177 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:44.026155    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7676-7cd785"
Sep 12 00:52:44.300515 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:44.300486    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7676-7cd785"]
Sep 12 00:52:44.302498 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:52:44.302473    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod064d5746_54f3_43c1_a138_a20f7f41d72c.slice/crio-b3636104fce1795a24341848c97a355d322555e13c2c16229c49efc083aa0b3c WatchSource:0}: Error finding container b3636104fce1795a24341848c97a355d322555e13c2c16229c49efc083aa0b3c: Status 404 returned error can't find the container with id b3636104fce1795a24341848c97a355d322555e13c2c16229c49efc083aa0b3c
Sep 12 00:52:45.019046 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:45.019008    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7676-7cd785" event={"ID":"064d5746-54f3-43c1-a138-a20f7f41d72c","Type":"ContainerStarted","Data":"e7d6447d4795a52d1e895d59686c0d163ee0c0790e9b1bff252b19873d5f4505"}
Sep 12 00:52:45.019514 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:45.019062    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7676-7cd785" event={"ID":"064d5746-54f3-43c1-a138-a20f7f41d72c","Type":"ContainerStarted","Data":"b3636104fce1795a24341848c97a355d322555e13c2c16229c49efc083aa0b3c"}
Sep 12 00:52:45.019514 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:45.019076    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7676-a482ff" podUID="d57e57e6-4ce3-4579-bc3d-8fba02271e14" containerName="test-curl-7676-a482ff" containerID="cri-o://748c303f947cad6c43a04cb6288407bdd8c8aa95a7ad04a2d2ba223c6bf19325" gracePeriod=2
Sep 12 00:52:45.037812 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:45.037755    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7676-7cd785" podStartSLOduration=2.037735311 podStartE2EDuration="2.037735311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:52:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:52:45.035736405 +0000 UTC m=+1401.709795265" watchObservedRunningTime="2026-09-12 00:52:45.037735311 +0000 UTC m=+1401.711794171"
Sep 12 00:52:45.133497 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:45.133465    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4"
Sep 12 00:52:45.133861 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:45.133812    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4"
Sep 12 00:52:45.149502 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:45.149473    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4"
Sep 12 00:52:45.196313 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:45.196272    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:45.210806 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:45.210783    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7676-640920"
Sep 12 00:52:45.378777 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:45.378691    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p8xdh\" (UniqueName: \"kubernetes.io/projected/6fd2725a-39d1-42bd-bbc4-da57f052e676-kube-api-access-p8xdh\") pod \"6fd2725a-39d1-42bd-bbc4-da57f052e676\" (UID: \"6fd2725a-39d1-42bd-bbc4-da57f052e676\") "
Sep 12 00:52:45.381043 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:45.381021    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fd2725a-39d1-42bd-bbc4-da57f052e676-kube-api-access-p8xdh" (OuterVolumeSpecName: "kube-api-access-p8xdh") pod "6fd2725a-39d1-42bd-bbc4-da57f052e676" (UID: "6fd2725a-39d1-42bd-bbc4-da57f052e676"). InnerVolumeSpecName "kube-api-access-p8xdh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:52:45.479951 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:45.479916    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8xdh\" (UniqueName: \"kubernetes.io/projected/6fd2725a-39d1-42bd-bbc4-da57f052e676-kube-api-access-p8xdh\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:52:45.606611 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:45.606556    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7676-7cd785"]
Sep 12 00:52:45.611456 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:45.611418    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7676-7cd785"]
Sep 12 00:52:45.726398 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:45.726364    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7676-68c33f"]
Sep 12 00:52:45.726822 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:45.726807    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="064d5746-54f3-43c1-a138-a20f7f41d72c" containerName="test-curl-7676-7cd785"
Sep 12 00:52:45.726822 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:45.726821    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="064d5746-54f3-43c1-a138-a20f7f41d72c" containerName="test-curl-7676-7cd785"
Sep 12 00:52:45.726932 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:45.726903    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="064d5746-54f3-43c1-a138-a20f7f41d72c" containerName="test-curl-7676-7cd785"
Sep 12 00:52:45.730034 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:45.730017    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7676-68c33f"
Sep 12 00:52:45.739920 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:45.739890    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fd2725a-39d1-42bd-bbc4-da57f052e676" path="/var/lib/kubelet/pods/6fd2725a-39d1-42bd-bbc4-da57f052e676/volumes"
Sep 12 00:52:45.740344 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:45.740326    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7676-68c33f"]
Sep 12 00:52:45.782352 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:45.782319    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lts8q\" (UniqueName: \"kubernetes.io/projected/f9569ab7-d305-4fcc-8958-ed2d492c63f1-kube-api-access-lts8q\") pod \"test-curl-7676-68c33f\" (UID: \"f9569ab7-d305-4fcc-8958-ed2d492c63f1\") " pod="opendatahub/test-curl-7676-68c33f"
Sep 12 00:52:45.883277 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:45.883242    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lts8q\" (UniqueName: \"kubernetes.io/projected/f9569ab7-d305-4fcc-8958-ed2d492c63f1-kube-api-access-lts8q\") pod \"test-curl-7676-68c33f\" (UID: \"f9569ab7-d305-4fcc-8958-ed2d492c63f1\") " pod="opendatahub/test-curl-7676-68c33f"
Sep 12 00:52:45.891835 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:45.891809    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lts8q\" (UniqueName: \"kubernetes.io/projected/f9569ab7-d305-4fcc-8958-ed2d492c63f1-kube-api-access-lts8q\") pod \"test-curl-7676-68c33f\" (UID: \"f9569ab7-d305-4fcc-8958-ed2d492c63f1\") " pod="opendatahub/test-curl-7676-68c33f"
Sep 12 00:52:45.906615 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:45.906585    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:46.025395 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:46.025298    2558 generic.go:356] "Generic (PLEG): container finished" podID="6fd2725a-39d1-42bd-bbc4-da57f052e676" containerID="fb1222aca7a690a4e7e7f394d2ac4ea3fc9b0fe08ce15d7705cf091bda52889c" exitCode=137
Sep 12 00:52:46.025395 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:46.025338    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7676-640920"
Sep 12 00:52:46.025395 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:46.025378    2558 scope.go:117] "RemoveContainer" containerID="fb1222aca7a690a4e7e7f394d2ac4ea3fc9b0fe08ce15d7705cf091bda52889c"
Sep 12 00:52:46.025928 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:52:46.025578    2558 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7676-7cd785" container="test-curl-7676-7cd785"
Sep 12 00:52:46.034542 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:46.034525    2558 scope.go:117] "RemoveContainer" containerID="fb1222aca7a690a4e7e7f394d2ac4ea3fc9b0fe08ce15d7705cf091bda52889c"
Sep 12 00:52:46.034815 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:52:46.034796    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb1222aca7a690a4e7e7f394d2ac4ea3fc9b0fe08ce15d7705cf091bda52889c\": container with ID starting with fb1222aca7a690a4e7e7f394d2ac4ea3fc9b0fe08ce15d7705cf091bda52889c not found: ID does not exist" containerID="fb1222aca7a690a4e7e7f394d2ac4ea3fc9b0fe08ce15d7705cf091bda52889c"
Sep 12 00:52:46.034889 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:46.034823    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb1222aca7a690a4e7e7f394d2ac4ea3fc9b0fe08ce15d7705cf091bda52889c"} err="failed to get container status \"fb1222aca7a690a4e7e7f394d2ac4ea3fc9b0fe08ce15d7705cf091bda52889c\": rpc error: code = NotFound desc = could not find container \"fb1222aca7a690a4e7e7f394d2ac4ea3fc9b0fe08ce15d7705cf091bda52889c\": container with ID starting with fb1222aca7a690a4e7e7f394d2ac4ea3fc9b0fe08ce15d7705cf091bda52889c not found: ID does not exist"
Sep 12 00:52:46.037376 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:46.037359    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4"
Sep 12 00:52:46.040216 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:46.040202    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7676-68c33f"
Sep 12 00:52:46.321749 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:46.321727    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7676-68c33f"]
Sep 12 00:52:46.324036 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:52:46.324007    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf9569ab7_d305_4fcc_8958_ed2d492c63f1.slice/crio-b3a65ac16866dbb632f9c13ae85d592c27bc75542677082477f15821bdc25569 WatchSource:0}: Error finding container b3a65ac16866dbb632f9c13ae85d592c27bc75542677082477f15821bdc25569: Status 404 returned error can't find the container with id b3a65ac16866dbb632f9c13ae85d592c27bc75542677082477f15821bdc25569
Sep 12 00:52:46.678429 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:46.678328    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:47.033923 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:47.033889    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7676-68c33f" event={"ID":"f9569ab7-d305-4fcc-8958-ed2d492c63f1","Type":"ContainerStarted","Data":"727dc91bf61df39233385d03fa33125b94c180fab34b5f1ab7402b85572fcbf4"}
Sep 12 00:52:47.034332 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:47.033940    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7676-68c33f" event={"ID":"f9569ab7-d305-4fcc-8958-ed2d492c63f1","Type":"ContainerStarted","Data":"b3a65ac16866dbb632f9c13ae85d592c27bc75542677082477f15821bdc25569"}
Sep 12 00:52:47.034332 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:47.034127    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7676-7cd785" podUID="064d5746-54f3-43c1-a138-a20f7f41d72c" containerName="test-curl-7676-7cd785" containerID="cri-o://e7d6447d4795a52d1e895d59686c0d163ee0c0790e9b1bff252b19873d5f4505" gracePeriod=2
Sep 12 00:52:47.050638 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:47.050597    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7676-68c33f" podStartSLOduration=2.050581036 podStartE2EDuration="2.050581036s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:52:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:52:47.049155266 +0000 UTC m=+1403.723214138" watchObservedRunningTime="2026-09-12 00:52:47.050581036 +0000 UTC m=+1403.724639894"
Sep 12 00:52:47.203913 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:47.203886    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7676-a482ff"
Sep 12 00:52:47.298488 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:47.298405    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-st7xz\" (UniqueName: \"kubernetes.io/projected/d57e57e6-4ce3-4579-bc3d-8fba02271e14-kube-api-access-st7xz\") pod \"d57e57e6-4ce3-4579-bc3d-8fba02271e14\" (UID: \"d57e57e6-4ce3-4579-bc3d-8fba02271e14\") "
Sep 12 00:52:47.300504 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:47.300471    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d57e57e6-4ce3-4579-bc3d-8fba02271e14-kube-api-access-st7xz" (OuterVolumeSpecName: "kube-api-access-st7xz") pod "d57e57e6-4ce3-4579-bc3d-8fba02271e14" (UID: "d57e57e6-4ce3-4579-bc3d-8fba02271e14"). InnerVolumeSpecName "kube-api-access-st7xz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:52:47.399454 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:47.399420    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-st7xz\" (UniqueName: \"kubernetes.io/projected/d57e57e6-4ce3-4579-bc3d-8fba02271e14-kube-api-access-st7xz\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:52:47.486101 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:47.486064    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:47.621049 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:47.620968    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7676-68c33f"]
Sep 12 00:52:47.633828 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:47.633796    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7676-68c33f"]
Sep 12 00:52:47.738823 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:47.738789    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d57e57e6-4ce3-4579-bc3d-8fba02271e14" path="/var/lib/kubelet/pods/d57e57e6-4ce3-4579-bc3d-8fba02271e14/volumes"
Sep 12 00:52:48.041360 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:48.041322    2558 generic.go:356] "Generic (PLEG): container finished" podID="d57e57e6-4ce3-4579-bc3d-8fba02271e14" containerID="748c303f947cad6c43a04cb6288407bdd8c8aa95a7ad04a2d2ba223c6bf19325" exitCode=137
Sep 12 00:52:48.041800 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:48.041379    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7676-a482ff"
Sep 12 00:52:48.041800 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:48.041432    2558 scope.go:117] "RemoveContainer" containerID="748c303f947cad6c43a04cb6288407bdd8c8aa95a7ad04a2d2ba223c6bf19325"
Sep 12 00:52:48.041930 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:52:48.041825    2558 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7676-68c33f" container="test-curl-7676-68c33f"
Sep 12 00:52:48.051027 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:48.051009    2558 scope.go:117] "RemoveContainer" containerID="748c303f947cad6c43a04cb6288407bdd8c8aa95a7ad04a2d2ba223c6bf19325"
Sep 12 00:52:48.051348 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:52:48.051327    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"748c303f947cad6c43a04cb6288407bdd8c8aa95a7ad04a2d2ba223c6bf19325\": container with ID starting with 748c303f947cad6c43a04cb6288407bdd8c8aa95a7ad04a2d2ba223c6bf19325 not found: ID does not exist" containerID="748c303f947cad6c43a04cb6288407bdd8c8aa95a7ad04a2d2ba223c6bf19325"
Sep 12 00:52:48.051415 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:48.051356    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"748c303f947cad6c43a04cb6288407bdd8c8aa95a7ad04a2d2ba223c6bf19325"} err="failed to get container status \"748c303f947cad6c43a04cb6288407bdd8c8aa95a7ad04a2d2ba223c6bf19325\": rpc error: code = NotFound desc = could not find container \"748c303f947cad6c43a04cb6288407bdd8c8aa95a7ad04a2d2ba223c6bf19325\": container with ID starting with 748c303f947cad6c43a04cb6288407bdd8c8aa95a7ad04a2d2ba223c6bf19325 not found: ID does not exist"
Sep 12 00:52:48.198927 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:48.198897    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:48.300415 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:48.300331    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:48.514556 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:48.514513    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:49.049992 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:49.049963    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7676-68c33f" podUID="f9569ab7-d305-4fcc-8958-ed2d492c63f1" containerName="test-curl-7676-68c33f" containerID="cri-o://727dc91bf61df39233385d03fa33125b94c180fab34b5f1ab7402b85572fcbf4" gracePeriod=2
Sep 12 00:52:49.090487 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:49.090448    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:49.198615 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:49.198580    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:49.222718 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:49.222692    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7676-7cd785"
Sep 12 00:52:49.317796 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:49.317711    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mqqq8\" (UniqueName: \"kubernetes.io/projected/064d5746-54f3-43c1-a138-a20f7f41d72c-kube-api-access-mqqq8\") pod \"064d5746-54f3-43c1-a138-a20f7f41d72c\" (UID: \"064d5746-54f3-43c1-a138-a20f7f41d72c\") "
Sep 12 00:52:49.319960 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:49.319938    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/064d5746-54f3-43c1-a138-a20f7f41d72c-kube-api-access-mqqq8" (OuterVolumeSpecName: "kube-api-access-mqqq8") pod "064d5746-54f3-43c1-a138-a20f7f41d72c" (UID: "064d5746-54f3-43c1-a138-a20f7f41d72c"). InnerVolumeSpecName "kube-api-access-mqqq8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:52:49.419167 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:49.419139    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqqq8\" (UniqueName: \"kubernetes.io/projected/064d5746-54f3-43c1-a138-a20f7f41d72c-kube-api-access-mqqq8\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:52:49.739184 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:49.739153    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="064d5746-54f3-43c1-a138-a20f7f41d72c" path="/var/lib/kubelet/pods/064d5746-54f3-43c1-a138-a20f7f41d72c/volumes"
Sep 12 00:52:49.976819 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:49.976788    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:50.056800 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:50.056717    2558 generic.go:356] "Generic (PLEG): container finished" podID="064d5746-54f3-43c1-a138-a20f7f41d72c" containerID="e7d6447d4795a52d1e895d59686c0d163ee0c0790e9b1bff252b19873d5f4505" exitCode=137
Sep 12 00:52:50.056800 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:50.056758    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7676-7cd785"
Sep 12 00:52:50.056800 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:50.056799    2558 scope.go:117] "RemoveContainer" containerID="e7d6447d4795a52d1e895d59686c0d163ee0c0790e9b1bff252b19873d5f4505"
Sep 12 00:52:50.067888 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:50.067407    2558 scope.go:117] "RemoveContainer" containerID="e7d6447d4795a52d1e895d59686c0d163ee0c0790e9b1bff252b19873d5f4505"
Sep 12 00:52:50.067888 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:52:50.067836    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7d6447d4795a52d1e895d59686c0d163ee0c0790e9b1bff252b19873d5f4505\": container with ID starting with e7d6447d4795a52d1e895d59686c0d163ee0c0790e9b1bff252b19873d5f4505 not found: ID does not exist" containerID="e7d6447d4795a52d1e895d59686c0d163ee0c0790e9b1bff252b19873d5f4505"
Sep 12 00:52:50.067888 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:50.067873    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7d6447d4795a52d1e895d59686c0d163ee0c0790e9b1bff252b19873d5f4505"} err="failed to get container status \"e7d6447d4795a52d1e895d59686c0d163ee0c0790e9b1bff252b19873d5f4505\": rpc error: code = NotFound desc = could not find container \"e7d6447d4795a52d1e895d59686c0d163ee0c0790e9b1bff252b19873d5f4505\": container with ID starting with e7d6447d4795a52d1e895d59686c0d163ee0c0790e9b1bff252b19873d5f4505 not found: ID does not exist"
Sep 12 00:52:50.662422 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:50.662386    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:51.235148 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:51.235125    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7676-68c33f"
Sep 12 00:52:51.338637 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:51.338541    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lts8q\" (UniqueName: \"kubernetes.io/projected/f9569ab7-d305-4fcc-8958-ed2d492c63f1-kube-api-access-lts8q\") pod \"f9569ab7-d305-4fcc-8958-ed2d492c63f1\" (UID: \"f9569ab7-d305-4fcc-8958-ed2d492c63f1\") "
Sep 12 00:52:51.340755 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:51.340722    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9569ab7-d305-4fcc-8958-ed2d492c63f1-kube-api-access-lts8q" (OuterVolumeSpecName: "kube-api-access-lts8q") pod "f9569ab7-d305-4fcc-8958-ed2d492c63f1" (UID: "f9569ab7-d305-4fcc-8958-ed2d492c63f1"). InnerVolumeSpecName "kube-api-access-lts8q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:52:51.439412 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:51.439376    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lts8q\" (UniqueName: \"kubernetes.io/projected/f9569ab7-d305-4fcc-8958-ed2d492c63f1-kube-api-access-lts8q\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:52:51.439703 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:51.439682    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:51.739209 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:51.739174    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9569ab7-d305-4fcc-8958-ed2d492c63f1" path="/var/lib/kubelet/pods/f9569ab7-d305-4fcc-8958-ed2d492c63f1/volumes"
Sep 12 00:52:51.979433 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:51.979397    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:52.070013 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:52.069920    2558 generic.go:356] "Generic (PLEG): container finished" podID="f9569ab7-d305-4fcc-8958-ed2d492c63f1" containerID="727dc91bf61df39233385d03fa33125b94c180fab34b5f1ab7402b85572fcbf4" exitCode=137
Sep 12 00:52:52.070013 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:52.069969    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7676-68c33f"
Sep 12 00:52:52.070277 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:52.070025    2558 scope.go:117] "RemoveContainer" containerID="727dc91bf61df39233385d03fa33125b94c180fab34b5f1ab7402b85572fcbf4"
Sep 12 00:52:52.078038 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:52.078010    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:52.079886 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:52.079860    2558 scope.go:117] "RemoveContainer" containerID="727dc91bf61df39233385d03fa33125b94c180fab34b5f1ab7402b85572fcbf4"
Sep 12 00:52:52.080154 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:52:52.080133    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"727dc91bf61df39233385d03fa33125b94c180fab34b5f1ab7402b85572fcbf4\": container with ID starting with 727dc91bf61df39233385d03fa33125b94c180fab34b5f1ab7402b85572fcbf4 not found: ID does not exist" containerID="727dc91bf61df39233385d03fa33125b94c180fab34b5f1ab7402b85572fcbf4"
Sep 12 00:52:52.080245 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:52.080164    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"727dc91bf61df39233385d03fa33125b94c180fab34b5f1ab7402b85572fcbf4"} err="failed to get container status \"727dc91bf61df39233385d03fa33125b94c180fab34b5f1ab7402b85572fcbf4\": rpc error: code = NotFound desc = could not find container \"727dc91bf61df39233385d03fa33125b94c180fab34b5f1ab7402b85572fcbf4\": container with ID starting with 727dc91bf61df39233385d03fa33125b94c180fab34b5f1ab7402b85572fcbf4 not found: ID does not exist"
Sep 12 00:52:52.392822 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:52.392729    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:53.314517 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:53.314481    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:54.089674 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:54.089559    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:54.794216 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:54.794172    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:55.631394 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:55.631353    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:56.342900 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:56.342863    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:56.371232 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:56.371200    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-81df64-5c8776d946-srz6k"]
Sep 12 00:52:56.371484 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:56.371458    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-81df64-5c8776d946-srz6k" podUID="b5cf8184-7399-4517-a934-cecdb05cb196" containerName="payload-pre-processing" containerID="cri-o://6bf055cb0e6766f45d677cb36107bc4251b3f72ad6e0a7541e01fff4e509a28e" gracePeriod=30
Sep 12 00:52:56.442714 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:56.442665    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:56.552706 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:56.552657    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:56.571154 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:56.570373    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc"]
Sep 12 00:52:56.571154 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:56.571138    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9569ab7-d305-4fcc-8958-ed2d492c63f1" containerName="test-curl-7676-68c33f"
Sep 12 00:52:56.571154 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:56.571158    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9569ab7-d305-4fcc-8958-ed2d492c63f1" containerName="test-curl-7676-68c33f"
Sep 12 00:52:56.571472 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:56.571262    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9569ab7-d305-4fcc-8958-ed2d492c63f1" containerName="test-curl-7676-68c33f"
Sep 12 00:52:56.575044 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:56.575016    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc"
Sep 12 00:52:56.578912 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:56.578888    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-5306e6ca-serving-cert\""
Sep 12 00:52:56.592423 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:56.592384    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc"]
Sep 12 00:52:56.639452 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:56.639424    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-81df64-5c8776d946-srz6k"
Sep 12 00:52:56.644217 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:56.644191    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:56.693997 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:56.693961    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d54hh\" (UniqueName: \"kubernetes.io/projected/b5cf8184-7399-4517-a934-cecdb05cb196-kube-api-access-d54hh\") pod \"b5cf8184-7399-4517-a934-cecdb05cb196\" (UID: \"b5cf8184-7399-4517-a934-cecdb05cb196\") "
Sep 12 00:52:56.694215 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:56.694195    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5cf8184-7399-4517-a934-cecdb05cb196-plugins-config-volume\") pod \"b5cf8184-7399-4517-a934-cecdb05cb196\" (UID: \"b5cf8184-7399-4517-a934-cecdb05cb196\") "
Sep 12 00:52:56.694435 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:56.694391    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxwzc\" (UniqueName: \"kubernetes.io/projected/6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0-kube-api-access-mxwzc\") pod \"maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc\" (UID: \"6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc"
Sep 12 00:52:56.694597 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:56.694517    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0-openshift-service-ca\") pod \"maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc\" (UID: \"6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc"
Sep 12 00:52:56.694661 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:56.694614    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0-metrics-certs\") pod \"maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc\" (UID: \"6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc"
Sep 12 00:52:56.694661 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:56.694619    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5cf8184-7399-4517-a934-cecdb05cb196-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b5cf8184-7399-4517-a934-cecdb05cb196" (UID: "b5cf8184-7399-4517-a934-cecdb05cb196"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:52:56.694769 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:56.694699    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0-maas-api-tls\") pod \"maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc\" (UID: \"6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc"
Sep 12 00:52:56.694858 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:56.694834    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5cf8184-7399-4517-a934-cecdb05cb196-plugins-config-volume\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:52:56.696292 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:56.696266    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5cf8184-7399-4517-a934-cecdb05cb196-kube-api-access-d54hh" (OuterVolumeSpecName: "kube-api-access-d54hh") pod "b5cf8184-7399-4517-a934-cecdb05cb196" (UID: "b5cf8184-7399-4517-a934-cecdb05cb196"). InnerVolumeSpecName "kube-api-access-d54hh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:52:56.796237 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:56.796191    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0-openshift-service-ca\") pod \"maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc\" (UID: \"6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc"
Sep 12 00:52:56.796462 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:56.796276    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0-metrics-certs\") pod \"maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc\" (UID: \"6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc"
Sep 12 00:52:56.796462 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:56.796308    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0-maas-api-tls\") pod \"maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc\" (UID: \"6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc"
Sep 12 00:52:56.796462 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:56.796371    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxwzc\" (UniqueName: \"kubernetes.io/projected/6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0-kube-api-access-mxwzc\") pod \"maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc\" (UID: \"6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc"
Sep 12 00:52:56.796462 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:56.796445    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d54hh\" (UniqueName: \"kubernetes.io/projected/b5cf8184-7399-4517-a934-cecdb05cb196-kube-api-access-d54hh\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:52:56.796679 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:52:56.796509    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-5306e6ca-serving-cert: secret "maas-api-e2e-mt-5306e6ca-serving-cert" not found
Sep 12 00:52:56.796679 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:52:56.796581    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0-maas-api-tls podName:6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0 nodeName:}" failed. No retries permitted until 2026-09-12 00:52:57.296558983 +0000 UTC m=+1413.970617824 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0-maas-api-tls") pod "maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc" (UID: "6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0") : secret "maas-api-e2e-mt-5306e6ca-serving-cert" not found
Sep 12 00:52:56.796949 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:56.796928    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0-openshift-service-ca\") pod \"maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc\" (UID: \"6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc"
Sep 12 00:52:56.798893 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:56.798865    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0-metrics-certs\") pod \"maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc\" (UID: \"6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc"
Sep 12 00:52:56.814938 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:56.814906    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxwzc\" (UniqueName: \"kubernetes.io/projected/6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0-kube-api-access-mxwzc\") pod \"maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc\" (UID: \"6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc"
Sep 12 00:52:56.863614 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:56.863584    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:56.963730 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:56.963701    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:57.098877 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:57.098844    2558 generic.go:356] "Generic (PLEG): container finished" podID="b5cf8184-7399-4517-a934-cecdb05cb196" containerID="6bf055cb0e6766f45d677cb36107bc4251b3f72ad6e0a7541e01fff4e509a28e" exitCode=0
Sep 12 00:52:57.099063 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:57.098893    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-81df64-5c8776d946-srz6k"
Sep 12 00:52:57.099063 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:57.098921    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-81df64-5c8776d946-srz6k" event={"ID":"b5cf8184-7399-4517-a934-cecdb05cb196","Type":"ContainerDied","Data":"6bf055cb0e6766f45d677cb36107bc4251b3f72ad6e0a7541e01fff4e509a28e"}
Sep 12 00:52:57.099063 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:57.098990    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-81df64-5c8776d946-srz6k" event={"ID":"b5cf8184-7399-4517-a934-cecdb05cb196","Type":"ContainerDied","Data":"b309c5db8b1cc6189fa39094e59bd5d7fdb2c6de7228d9df4f77fb2433964aca"}
Sep 12 00:52:57.099063 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:57.099022    2558 scope.go:117] "RemoveContainer" containerID="6bf055cb0e6766f45d677cb36107bc4251b3f72ad6e0a7541e01fff4e509a28e"
Sep 12 00:52:57.109652 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:57.109631    2558 scope.go:117] "RemoveContainer" containerID="6bf055cb0e6766f45d677cb36107bc4251b3f72ad6e0a7541e01fff4e509a28e"
Sep 12 00:52:57.109913 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:52:57.109894    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bf055cb0e6766f45d677cb36107bc4251b3f72ad6e0a7541e01fff4e509a28e\": container with ID starting with 6bf055cb0e6766f45d677cb36107bc4251b3f72ad6e0a7541e01fff4e509a28e not found: ID does not exist" containerID="6bf055cb0e6766f45d677cb36107bc4251b3f72ad6e0a7541e01fff4e509a28e"
Sep 12 00:52:57.109975 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:57.109928    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bf055cb0e6766f45d677cb36107bc4251b3f72ad6e0a7541e01fff4e509a28e"} err="failed to get container status \"6bf055cb0e6766f45d677cb36107bc4251b3f72ad6e0a7541e01fff4e509a28e\": rpc error: code = NotFound desc = could not find container \"6bf055cb0e6766f45d677cb36107bc4251b3f72ad6e0a7541e01fff4e509a28e\": container with ID starting with 6bf055cb0e6766f45d677cb36107bc4251b3f72ad6e0a7541e01fff4e509a28e not found: ID does not exist"
Sep 12 00:52:57.124234 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:57.124211    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-81df64-5c8776d946-srz6k"]
Sep 12 00:52:57.130903 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:57.130883    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-81df64-5c8776d946-srz6k"]
Sep 12 00:52:57.230414 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:57.230334    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:57.301450 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:57.301414    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0-maas-api-tls\") pod \"maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc\" (UID: \"6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc"
Sep 12 00:52:57.303899 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:57.303874    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0-maas-api-tls\") pod \"maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc\" (UID: \"6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc"
Sep 12 00:52:57.320634 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:57.320609    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:57.341397 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:57.341376    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5306e6ca-7986884b4b-f9k62"]
Sep 12 00:52:57.341818 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:57.341805    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5cf8184-7399-4517-a934-cecdb05cb196" containerName="payload-pre-processing"
Sep 12 00:52:57.341873 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:57.341821    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5cf8184-7399-4517-a934-cecdb05cb196" containerName="payload-pre-processing"
Sep 12 00:52:57.341915 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:57.341897    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5cf8184-7399-4517-a934-cecdb05cb196" containerName="payload-pre-processing"
Sep 12 00:52:57.344818 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:57.344804    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-5306e6ca-7986884b4b-f9k62"
Sep 12 00:52:57.347382 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:57.347363    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-5306e6ca-dockercfg-ccsj6\""
Sep 12 00:52:57.347474 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:57.347406    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-5306e6ca\""
Sep 12 00:52:57.356223 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:57.356197    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5306e6ca-7986884b4b-f9k62"]
Sep 12 00:52:57.402869 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:57.402844    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qp2zd\" (UniqueName: \"kubernetes.io/projected/0e7c3936-0cce-4fa0-8d12-4c41e89203cd-kube-api-access-qp2zd\") pod \"payload-pre-processing-e2e-mt-5306e6ca-7986884b4b-f9k62\" (UID: \"0e7c3936-0cce-4fa0-8d12-4c41e89203cd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5306e6ca-7986884b4b-f9k62"
Sep 12 00:52:57.403022 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:57.402916    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e7c3936-0cce-4fa0-8d12-4c41e89203cd-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-5306e6ca-7986884b4b-f9k62\" (UID: \"0e7c3936-0cce-4fa0-8d12-4c41e89203cd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5306e6ca-7986884b4b-f9k62"
Sep 12 00:52:57.487963 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:57.487881    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc"
Sep 12 00:52:57.503890 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:57.503855    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qp2zd\" (UniqueName: \"kubernetes.io/projected/0e7c3936-0cce-4fa0-8d12-4c41e89203cd-kube-api-access-qp2zd\") pod \"payload-pre-processing-e2e-mt-5306e6ca-7986884b4b-f9k62\" (UID: \"0e7c3936-0cce-4fa0-8d12-4c41e89203cd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5306e6ca-7986884b4b-f9k62"
Sep 12 00:52:57.504024 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:57.503994    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e7c3936-0cce-4fa0-8d12-4c41e89203cd-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-5306e6ca-7986884b4b-f9k62\" (UID: \"0e7c3936-0cce-4fa0-8d12-4c41e89203cd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5306e6ca-7986884b4b-f9k62"
Sep 12 00:52:57.504659 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:57.504637    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e7c3936-0cce-4fa0-8d12-4c41e89203cd-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-5306e6ca-7986884b4b-f9k62\" (UID: \"0e7c3936-0cce-4fa0-8d12-4c41e89203cd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5306e6ca-7986884b4b-f9k62"
Sep 12 00:52:57.512709 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:57.512680    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qp2zd\" (UniqueName: \"kubernetes.io/projected/0e7c3936-0cce-4fa0-8d12-4c41e89203cd-kube-api-access-qp2zd\") pod \"payload-pre-processing-e2e-mt-5306e6ca-7986884b4b-f9k62\" (UID: \"0e7c3936-0cce-4fa0-8d12-4c41e89203cd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5306e6ca-7986884b4b-f9k62"
Sep 12 00:52:57.656525 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:57.656496    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-5306e6ca-7986884b4b-f9k62"
Sep 12 00:52:57.682185 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:57.680690    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:57.739437 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:57.739371    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5cf8184-7399-4517-a934-cecdb05cb196" path="/var/lib/kubelet/pods/b5cf8184-7399-4517-a934-cecdb05cb196/volumes"
Sep 12 00:52:57.773340 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:57.773311    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc"]
Sep 12 00:52:57.775157 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:52:57.775118    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ef36dc5_ee4c_4882_bbb3_e08677dfc7a0.slice/crio-b843f99c690571890097731f16cbce74d4d77e6b2b6bb0787d619e9bd5fe76e1 WatchSource:0}: Error finding container b843f99c690571890097731f16cbce74d4d77e6b2b6bb0787d619e9bd5fe76e1: Status 404 returned error can't find the container with id b843f99c690571890097731f16cbce74d4d77e6b2b6bb0787d619e9bd5fe76e1
Sep 12 00:52:57.937240 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:57.937213    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5306e6ca-7986884b4b-f9k62"]
Sep 12 00:52:57.938146 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:52:57.938122    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e7c3936_0cce_4fa0_8d12_4c41e89203cd.slice/crio-09b0934a92bf6ddf54165237a94090a9d29af393fa807b326c709d7baf12cfed WatchSource:0}: Error finding container 09b0934a92bf6ddf54165237a94090a9d29af393fa807b326c709d7baf12cfed: Status 404 returned error can't find the container with id 09b0934a92bf6ddf54165237a94090a9d29af393fa807b326c709d7baf12cfed
Sep 12 00:52:58.106937 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:58.106905    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc" event={"ID":"6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0","Type":"ContainerStarted","Data":"b843f99c690571890097731f16cbce74d4d77e6b2b6bb0787d619e9bd5fe76e1"}
Sep 12 00:52:58.108823 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:58.108791    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-5306e6ca-7986884b4b-f9k62" event={"ID":"0e7c3936-0cce-4fa0-8d12-4c41e89203cd","Type":"ContainerStarted","Data":"ac534fe28f3d86ea179c39824a6d015a19c777b7ce29cab60856bff25744f3e8"}
Sep 12 00:52:58.108935 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:58.108852    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-5306e6ca-7986884b4b-f9k62" event={"ID":"0e7c3936-0cce-4fa0-8d12-4c41e89203cd","Type":"ContainerStarted","Data":"09b0934a92bf6ddf54165237a94090a9d29af393fa807b326c709d7baf12cfed"}
Sep 12 00:52:58.108935 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:58.108902    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-5306e6ca-7986884b4b-f9k62"
Sep 12 00:52:58.126582 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:58.126539    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-5306e6ca-7986884b4b-f9k62" podStartSLOduration=1.126522907 podStartE2EDuration="1.126522907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:52:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:52:58.126025617 +0000 UTC m=+1414.800084475" watchObservedRunningTime="2026-09-12 00:52:58.126522907 +0000 UTC m=+1414.800581759"
Sep 12 00:52:58.564363 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:58.564323    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:59.117395 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:59.117359    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc" event={"ID":"6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0","Type":"ContainerStarted","Data":"40dd363059d107a27779b9aded7388f0125d54026b2e11ced8dc87aa1a200a4a"}
Sep 12 00:52:59.136559 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:59.136494    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc" podStartSLOduration=2.817448347 podStartE2EDuration="3.136472057s" totalImagesPullingTime="319.02371ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:52:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:52:59.13524173 +0000 UTC m=+1415.809300603" watchObservedRunningTime="2026-09-12 00:52:59.136472057 +0000 UTC m=+1415.810530916"
Sep 12 00:52:59.271534 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:59.271477    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:52:59.970855 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:52:59.970818    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:00.123186 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:00.123145    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc"
Sep 12 00:53:00.742281 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:00.742251    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:01.467372 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:01.467340    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:02.230372 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:02.230336    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:03.154847 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:03.154813    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:03.218711 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:03.218659    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:03.502525 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:03.502489    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:04.078395 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:04.078365    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:04.806401 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:04.806361    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:05.673892 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:05.673853    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:06.134336 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:06.134308    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc"
Sep 12 00:53:06.359834 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:06.359795    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:07.142393 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:07.142359    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:08.070872 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:08.070836    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:08.772507 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:08.772467    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:09.119922 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:09.119828    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-5306e6ca-7986884b4b-f9k62"
Sep 12 00:53:09.502049 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:09.502015    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:10.307934 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:10.307898    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:10.399237 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:10.399200    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:10.587195 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:10.587108    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:11.704452 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:11.704401    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:11.807529 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:11.807496    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:12.221347 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:12.221310    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:13.032401 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:13.032365    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:13.708080 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:13.708047    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:14.321746 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:14.321716    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:14.348231 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:14.348199    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-bb6e74-59d9668859-c96ng"]
Sep 12 00:53:14.348489 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:14.348455    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-bb6e74-59d9668859-c96ng" podUID="975e8170-7e78-40dd-a439-a24bd4a50165" containerName="payload-processing" containerID="cri-o://c9b2ef00ddcf4b89f3071df5ab15fcfe219449131a929e79a902e6ac5ab2b4db" gracePeriod=30
Sep 12 00:53:14.356456 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:14.356429    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-bb6e74-6dfbdcd5d-bmst9"]
Sep 12 00:53:14.356706 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:14.356660    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bb6e74-6dfbdcd5d-bmst9" podUID="e38f1710-8367-4e0d-a2d5-e5b277194c2a" containerName="payload-pre-processing" containerID="cri-o://226007b6af187512bfbaf9f87fbd1f2d3b665a47cf6bf6e8cc14bc7b9d8897ec" gracePeriod=30
Sep 12 00:53:14.391922 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:14.391893    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:14.455684 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:14.455652    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:14.550512 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:14.550476    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:14.617674 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:14.617629    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:14.629706 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:14.629679    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-bb6e74-59d9668859-c96ng"
Sep 12 00:53:14.669753 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:14.669720    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bb6e74-6dfbdcd5d-bmst9"
Sep 12 00:53:14.793214 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:14.793178    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5hrx7\" (UniqueName: \"kubernetes.io/projected/e38f1710-8367-4e0d-a2d5-e5b277194c2a-kube-api-access-5hrx7\") pod \"e38f1710-8367-4e0d-a2d5-e5b277194c2a\" (UID: \"e38f1710-8367-4e0d-a2d5-e5b277194c2a\") "
Sep 12 00:53:14.793414 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:14.793290    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e38f1710-8367-4e0d-a2d5-e5b277194c2a-plugins-config-volume\") pod \"e38f1710-8367-4e0d-a2d5-e5b277194c2a\" (UID: \"e38f1710-8367-4e0d-a2d5-e5b277194c2a\") "
Sep 12 00:53:14.793414 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:14.793348    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kgjwc\" (UniqueName: \"kubernetes.io/projected/975e8170-7e78-40dd-a439-a24bd4a50165-kube-api-access-kgjwc\") pod \"975e8170-7e78-40dd-a439-a24bd4a50165\" (UID: \"975e8170-7e78-40dd-a439-a24bd4a50165\") "
Sep 12 00:53:14.793414 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:14.793408    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/975e8170-7e78-40dd-a439-a24bd4a50165-plugins-config-volume\") pod \"975e8170-7e78-40dd-a439-a24bd4a50165\" (UID: \"975e8170-7e78-40dd-a439-a24bd4a50165\") "
Sep 12 00:53:14.793857 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:14.793790    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e38f1710-8367-4e0d-a2d5-e5b277194c2a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e38f1710-8367-4e0d-a2d5-e5b277194c2a" (UID: "e38f1710-8367-4e0d-a2d5-e5b277194c2a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:53:14.794528 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:14.794121    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/975e8170-7e78-40dd-a439-a24bd4a50165-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "975e8170-7e78-40dd-a439-a24bd4a50165" (UID: "975e8170-7e78-40dd-a439-a24bd4a50165"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:53:14.796647 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:14.796618    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e38f1710-8367-4e0d-a2d5-e5b277194c2a-kube-api-access-5hrx7" (OuterVolumeSpecName: "kube-api-access-5hrx7") pod "e38f1710-8367-4e0d-a2d5-e5b277194c2a" (UID: "e38f1710-8367-4e0d-a2d5-e5b277194c2a"). InnerVolumeSpecName "kube-api-access-5hrx7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:53:14.796888 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:14.796859    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/975e8170-7e78-40dd-a439-a24bd4a50165-kube-api-access-kgjwc" (OuterVolumeSpecName: "kube-api-access-kgjwc") pod "975e8170-7e78-40dd-a439-a24bd4a50165" (UID: "975e8170-7e78-40dd-a439-a24bd4a50165"). InnerVolumeSpecName "kube-api-access-kgjwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:53:14.802332 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:14.802306    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2"]
Sep 12 00:53:14.826412 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:14.826340    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:14.895056 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:14.895010    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/975e8170-7e78-40dd-a439-a24bd4a50165-plugins-config-volume\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:53:14.895240 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:14.895142    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5hrx7\" (UniqueName: \"kubernetes.io/projected/e38f1710-8367-4e0d-a2d5-e5b277194c2a-kube-api-access-5hrx7\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:53:14.895240 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:14.895167    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e38f1710-8367-4e0d-a2d5-e5b277194c2a-plugins-config-volume\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:53:14.895240 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:14.895182    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgjwc\" (UniqueName: \"kubernetes.io/projected/975e8170-7e78-40dd-a439-a24bd4a50165-kube-api-access-kgjwc\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:53:14.925135 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:14.925069    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:15.026312 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:15.026274    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:15.214364 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:15.214332    2558 generic.go:356] "Generic (PLEG): container finished" podID="e38f1710-8367-4e0d-a2d5-e5b277194c2a" containerID="226007b6af187512bfbaf9f87fbd1f2d3b665a47cf6bf6e8cc14bc7b9d8897ec" exitCode=0
Sep 12 00:53:15.214562 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:15.214386    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bb6e74-6dfbdcd5d-bmst9"
Sep 12 00:53:15.214562 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:15.214415    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bb6e74-6dfbdcd5d-bmst9" event={"ID":"e38f1710-8367-4e0d-a2d5-e5b277194c2a","Type":"ContainerDied","Data":"226007b6af187512bfbaf9f87fbd1f2d3b665a47cf6bf6e8cc14bc7b9d8897ec"}
Sep 12 00:53:15.214562 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:15.214480    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bb6e74-6dfbdcd5d-bmst9" event={"ID":"e38f1710-8367-4e0d-a2d5-e5b277194c2a","Type":"ContainerDied","Data":"69d98678484002ad75d2963e88da30bb839224b3557c3864fa38f9c3d6d27243"}
Sep 12 00:53:15.214562 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:15.214517    2558 scope.go:117] "RemoveContainer" containerID="226007b6af187512bfbaf9f87fbd1f2d3b665a47cf6bf6e8cc14bc7b9d8897ec"
Sep 12 00:53:15.215869 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:15.215850    2558 generic.go:356] "Generic (PLEG): container finished" podID="975e8170-7e78-40dd-a439-a24bd4a50165" containerID="c9b2ef00ddcf4b89f3071df5ab15fcfe219449131a929e79a902e6ac5ab2b4db" exitCode=0
Sep 12 00:53:15.215999 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:15.215901    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-bb6e74-59d9668859-c96ng"
Sep 12 00:53:15.215999 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:15.215941    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-bb6e74-59d9668859-c96ng" event={"ID":"975e8170-7e78-40dd-a439-a24bd4a50165","Type":"ContainerDied","Data":"c9b2ef00ddcf4b89f3071df5ab15fcfe219449131a929e79a902e6ac5ab2b4db"}
Sep 12 00:53:15.215999 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:15.215982    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-bb6e74-59d9668859-c96ng" event={"ID":"975e8170-7e78-40dd-a439-a24bd4a50165","Type":"ContainerDied","Data":"de6456555c7bc6e67182e681ad6ca5ead22ce91706f6aba022174fb05eb36806"}
Sep 12 00:53:15.224125 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:15.224105    2558 scope.go:117] "RemoveContainer" containerID="226007b6af187512bfbaf9f87fbd1f2d3b665a47cf6bf6e8cc14bc7b9d8897ec"
Sep 12 00:53:15.224408 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:53:15.224391    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"226007b6af187512bfbaf9f87fbd1f2d3b665a47cf6bf6e8cc14bc7b9d8897ec\": container with ID starting with 226007b6af187512bfbaf9f87fbd1f2d3b665a47cf6bf6e8cc14bc7b9d8897ec not found: ID does not exist" containerID="226007b6af187512bfbaf9f87fbd1f2d3b665a47cf6bf6e8cc14bc7b9d8897ec"
Sep 12 00:53:15.224460 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:15.224418    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"226007b6af187512bfbaf9f87fbd1f2d3b665a47cf6bf6e8cc14bc7b9d8897ec"} err="failed to get container status \"226007b6af187512bfbaf9f87fbd1f2d3b665a47cf6bf6e8cc14bc7b9d8897ec\": rpc error: code = NotFound desc = could not find container \"226007b6af187512bfbaf9f87fbd1f2d3b665a47cf6bf6e8cc14bc7b9d8897ec\": container with ID starting with 226007b6af187512bfbaf9f87fbd1f2d3b665a47cf6bf6e8cc14bc7b9d8897ec not found: ID does not exist"
Sep 12 00:53:15.224460 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:15.224434    2558 scope.go:117] "RemoveContainer" containerID="c9b2ef00ddcf4b89f3071df5ab15fcfe219449131a929e79a902e6ac5ab2b4db"
Sep 12 00:53:15.232842 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:15.232823    2558 scope.go:117] "RemoveContainer" containerID="c9b2ef00ddcf4b89f3071df5ab15fcfe219449131a929e79a902e6ac5ab2b4db"
Sep 12 00:53:15.233109 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:53:15.233075    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9b2ef00ddcf4b89f3071df5ab15fcfe219449131a929e79a902e6ac5ab2b4db\": container with ID starting with c9b2ef00ddcf4b89f3071df5ab15fcfe219449131a929e79a902e6ac5ab2b4db not found: ID does not exist" containerID="c9b2ef00ddcf4b89f3071df5ab15fcfe219449131a929e79a902e6ac5ab2b4db"
Sep 12 00:53:15.233166 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:15.233119    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9b2ef00ddcf4b89f3071df5ab15fcfe219449131a929e79a902e6ac5ab2b4db"} err="failed to get container status \"c9b2ef00ddcf4b89f3071df5ab15fcfe219449131a929e79a902e6ac5ab2b4db\": rpc error: code = NotFound desc = could not find container \"c9b2ef00ddcf4b89f3071df5ab15fcfe219449131a929e79a902e6ac5ab2b4db\": container with ID starting with c9b2ef00ddcf4b89f3071df5ab15fcfe219449131a929e79a902e6ac5ab2b4db not found: ID does not exist"
Sep 12 00:53:15.240948 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:15.240925    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-bb6e74-59d9668859-c96ng"]
Sep 12 00:53:15.249746 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:15.249726    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-bb6e74-59d9668859-c96ng"]
Sep 12 00:53:15.265054 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:15.265028    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-bb6e74-6dfbdcd5d-bmst9"]
Sep 12 00:53:15.268682 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:15.268661    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-bb6e74-6dfbdcd5d-bmst9"]
Sep 12 00:53:15.668156 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:15.668034    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4"]
Sep 12 00:53:15.715973 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:15.715934    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:15.739183 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:15.739155    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="975e8170-7e78-40dd-a439-a24bd4a50165" path="/var/lib/kubelet/pods/975e8170-7e78-40dd-a439-a24bd4a50165/volumes"
Sep 12 00:53:15.739537 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:15.739525    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e38f1710-8367-4e0d-a2d5-e5b277194c2a" path="/var/lib/kubelet/pods/e38f1710-8367-4e0d-a2d5-e5b277194c2a/volumes"
Sep 12 00:53:15.816040 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:15.815991    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:16.580180 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:16.580145    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:17.276388 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:17.276359    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:18.127765 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:18.127730    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:18.987895 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:18.987853    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:19.668959 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:19.668917    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:19.983080 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:19.983045    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:23.649774 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:23.649739    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:23.750560 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:23.750524    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:29.823298 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:29.823254    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2" podUID="81b591cf-5ed8-4327-91a0-52e95b76c971" containerName="main" containerID="cri-o://d61242502e7bd8ef795b43853004221779bb031866146c4dde2977a1924b0faa" gracePeriod=45
Sep 12 00:53:30.085671 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:30.085612    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2"
Sep 12 00:53:30.149333 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:30.149305    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6t89b\" (UniqueName: \"kubernetes.io/projected/81b591cf-5ed8-4327-91a0-52e95b76c971-kube-api-access-6t89b\") pod \"81b591cf-5ed8-4327-91a0-52e95b76c971\" (UID: \"81b591cf-5ed8-4327-91a0-52e95b76c971\") "
Sep 12 00:53:30.149515 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:30.149376    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81b591cf-5ed8-4327-91a0-52e95b76c971-tls-certs\") pod \"81b591cf-5ed8-4327-91a0-52e95b76c971\" (UID: \"81b591cf-5ed8-4327-91a0-52e95b76c971\") "
Sep 12 00:53:30.149515 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:30.149412    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81b591cf-5ed8-4327-91a0-52e95b76c971-tmp-dir\") pod \"81b591cf-5ed8-4327-91a0-52e95b76c971\" (UID: \"81b591cf-5ed8-4327-91a0-52e95b76c971\") "
Sep 12 00:53:30.149515 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:30.149432    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81b591cf-5ed8-4327-91a0-52e95b76c971-model-cache\") pod \"81b591cf-5ed8-4327-91a0-52e95b76c971\" (UID: \"81b591cf-5ed8-4327-91a0-52e95b76c971\") "
Sep 12 00:53:30.149515 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:30.149495    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81b591cf-5ed8-4327-91a0-52e95b76c971-home\") pod \"81b591cf-5ed8-4327-91a0-52e95b76c971\" (UID: \"81b591cf-5ed8-4327-91a0-52e95b76c971\") "
Sep 12 00:53:30.149710 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:30.149539    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81b591cf-5ed8-4327-91a0-52e95b76c971-dshm\") pod \"81b591cf-5ed8-4327-91a0-52e95b76c971\" (UID: \"81b591cf-5ed8-4327-91a0-52e95b76c971\") "
Sep 12 00:53:30.149768 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:30.149712    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81b591cf-5ed8-4327-91a0-52e95b76c971-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "81b591cf-5ed8-4327-91a0-52e95b76c971" (UID: "81b591cf-5ed8-4327-91a0-52e95b76c971"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:53:30.149768 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:30.149754    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81b591cf-5ed8-4327-91a0-52e95b76c971-model-cache" (OuterVolumeSpecName: "model-cache") pod "81b591cf-5ed8-4327-91a0-52e95b76c971" (UID: "81b591cf-5ed8-4327-91a0-52e95b76c971"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:53:30.149883 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:30.149785    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81b591cf-5ed8-4327-91a0-52e95b76c971-home" (OuterVolumeSpecName: "home") pod "81b591cf-5ed8-4327-91a0-52e95b76c971" (UID: "81b591cf-5ed8-4327-91a0-52e95b76c971"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:53:30.150021 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:30.150004    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81b591cf-5ed8-4327-91a0-52e95b76c971-tmp-dir\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:53:30.150076 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:30.150027    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81b591cf-5ed8-4327-91a0-52e95b76c971-model-cache\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:53:30.150076 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:30.150044    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81b591cf-5ed8-4327-91a0-52e95b76c971-home\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:53:30.151866 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:30.151837    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81b591cf-5ed8-4327-91a0-52e95b76c971-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "81b591cf-5ed8-4327-91a0-52e95b76c971" (UID: "81b591cf-5ed8-4327-91a0-52e95b76c971"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:53:30.151866 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:30.151849    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81b591cf-5ed8-4327-91a0-52e95b76c971-kube-api-access-6t89b" (OuterVolumeSpecName: "kube-api-access-6t89b") pod "81b591cf-5ed8-4327-91a0-52e95b76c971" (UID: "81b591cf-5ed8-4327-91a0-52e95b76c971"). InnerVolumeSpecName "kube-api-access-6t89b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:53:30.152019 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:30.151842    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81b591cf-5ed8-4327-91a0-52e95b76c971-dshm" (OuterVolumeSpecName: "dshm") pod "81b591cf-5ed8-4327-91a0-52e95b76c971" (UID: "81b591cf-5ed8-4327-91a0-52e95b76c971"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:53:30.251258 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:30.251223    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81b591cf-5ed8-4327-91a0-52e95b76c971-tls-certs\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:53:30.251258 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:30.251250    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81b591cf-5ed8-4327-91a0-52e95b76c971-dshm\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:53:30.251258 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:30.251260    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6t89b\" (UniqueName: \"kubernetes.io/projected/81b591cf-5ed8-4327-91a0-52e95b76c971-kube-api-access-6t89b\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:53:30.299573 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:30.299538    2558 generic.go:356] "Generic (PLEG): container finished" podID="81b591cf-5ed8-4327-91a0-52e95b76c971" containerID="d61242502e7bd8ef795b43853004221779bb031866146c4dde2977a1924b0faa" exitCode=0
Sep 12 00:53:30.299753 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:30.299582    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2"
Sep 12 00:53:30.299753 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:30.299639    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2" event={"ID":"81b591cf-5ed8-4327-91a0-52e95b76c971","Type":"ContainerDied","Data":"d61242502e7bd8ef795b43853004221779bb031866146c4dde2977a1924b0faa"}
Sep 12 00:53:30.299753 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:30.299685    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2" event={"ID":"81b591cf-5ed8-4327-91a0-52e95b76c971","Type":"ContainerDied","Data":"1f330b916c2ef2ef199c49724f673454318aed98f315066c2e7c9ba7f05783f4"}
Sep 12 00:53:30.299753 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:30.299710    2558 scope.go:117] "RemoveContainer" containerID="d61242502e7bd8ef795b43853004221779bb031866146c4dde2977a1924b0faa"
Sep 12 00:53:30.309067 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:30.309045    2558 scope.go:117] "RemoveContainer" containerID="d61242502e7bd8ef795b43853004221779bb031866146c4dde2977a1924b0faa"
Sep 12 00:53:30.309397 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:53:30.309377    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d61242502e7bd8ef795b43853004221779bb031866146c4dde2977a1924b0faa\": container with ID starting with d61242502e7bd8ef795b43853004221779bb031866146c4dde2977a1924b0faa not found: ID does not exist" containerID="d61242502e7bd8ef795b43853004221779bb031866146c4dde2977a1924b0faa"
Sep 12 00:53:30.309465 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:30.309409    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d61242502e7bd8ef795b43853004221779bb031866146c4dde2977a1924b0faa"} err="failed to get container status \"d61242502e7bd8ef795b43853004221779bb031866146c4dde2977a1924b0faa\": rpc error: code = NotFound desc = could not find container \"d61242502e7bd8ef795b43853004221779bb031866146c4dde2977a1924b0faa\": container with ID starting with d61242502e7bd8ef795b43853004221779bb031866146c4dde2977a1924b0faa not found: ID does not exist"
Sep 12 00:53:30.320918 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:30.320889    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2"]
Sep 12 00:53:30.327663 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:30.327641    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-bb6e74/test-model-bb6e74-kserve-6996fcf55c-c87x2"]
Sep 12 00:53:30.688812 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:30.688767    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4" podUID="0eb8255b-f9fa-4570-a417-17c1845bcffc" containerName="main" containerID="cri-o://adc75f1c8f7d5aa6cbd9db50e9dbdf180cde0bc6a9e246fc75b2a61a93fc6543" gracePeriod=45
Sep 12 00:53:30.944275 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:30.944218    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4"
Sep 12 00:53:31.060252 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:31.060215    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0eb8255b-f9fa-4570-a417-17c1845bcffc-dshm\") pod \"0eb8255b-f9fa-4570-a417-17c1845bcffc\" (UID: \"0eb8255b-f9fa-4570-a417-17c1845bcffc\") "
Sep 12 00:53:31.060252 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:31.060253    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffr8r\" (UniqueName: \"kubernetes.io/projected/0eb8255b-f9fa-4570-a417-17c1845bcffc-kube-api-access-ffr8r\") pod \"0eb8255b-f9fa-4570-a417-17c1845bcffc\" (UID: \"0eb8255b-f9fa-4570-a417-17c1845bcffc\") "
Sep 12 00:53:31.060506 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:31.060279    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0eb8255b-f9fa-4570-a417-17c1845bcffc-model-cache\") pod \"0eb8255b-f9fa-4570-a417-17c1845bcffc\" (UID: \"0eb8255b-f9fa-4570-a417-17c1845bcffc\") "
Sep 12 00:53:31.060506 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:31.060328    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0eb8255b-f9fa-4570-a417-17c1845bcffc-home\") pod \"0eb8255b-f9fa-4570-a417-17c1845bcffc\" (UID: \"0eb8255b-f9fa-4570-a417-17c1845bcffc\") "
Sep 12 00:53:31.060506 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:31.060346    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0eb8255b-f9fa-4570-a417-17c1845bcffc-tmp-dir\") pod \"0eb8255b-f9fa-4570-a417-17c1845bcffc\" (UID: \"0eb8255b-f9fa-4570-a417-17c1845bcffc\") "
Sep 12 00:53:31.060506 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:31.060424    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0eb8255b-f9fa-4570-a417-17c1845bcffc-tls-certs\") pod \"0eb8255b-f9fa-4570-a417-17c1845bcffc\" (UID: \"0eb8255b-f9fa-4570-a417-17c1845bcffc\") "
Sep 12 00:53:31.060703 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:31.060621    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0eb8255b-f9fa-4570-a417-17c1845bcffc-home" (OuterVolumeSpecName: "home") pod "0eb8255b-f9fa-4570-a417-17c1845bcffc" (UID: "0eb8255b-f9fa-4570-a417-17c1845bcffc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:53:31.060703 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:31.060636    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0eb8255b-f9fa-4570-a417-17c1845bcffc-model-cache" (OuterVolumeSpecName: "model-cache") pod "0eb8255b-f9fa-4570-a417-17c1845bcffc" (UID: "0eb8255b-f9fa-4570-a417-17c1845bcffc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:53:31.060703 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:31.060657    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0eb8255b-f9fa-4570-a417-17c1845bcffc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0eb8255b-f9fa-4570-a417-17c1845bcffc" (UID: "0eb8255b-f9fa-4570-a417-17c1845bcffc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:53:31.060826 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:31.060814    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0eb8255b-f9fa-4570-a417-17c1845bcffc-model-cache\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:53:31.060878 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:31.060831    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0eb8255b-f9fa-4570-a417-17c1845bcffc-home\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:53:31.060878 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:31.060844    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0eb8255b-f9fa-4570-a417-17c1845bcffc-tmp-dir\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:53:31.062739 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:31.062712    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0eb8255b-f9fa-4570-a417-17c1845bcffc-dshm" (OuterVolumeSpecName: "dshm") pod "0eb8255b-f9fa-4570-a417-17c1845bcffc" (UID: "0eb8255b-f9fa-4570-a417-17c1845bcffc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:53:31.062861 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:31.062737    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0eb8255b-f9fa-4570-a417-17c1845bcffc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0eb8255b-f9fa-4570-a417-17c1845bcffc" (UID: "0eb8255b-f9fa-4570-a417-17c1845bcffc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:53:31.062861 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:31.062808    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0eb8255b-f9fa-4570-a417-17c1845bcffc-kube-api-access-ffr8r" (OuterVolumeSpecName: "kube-api-access-ffr8r") pod "0eb8255b-f9fa-4570-a417-17c1845bcffc" (UID: "0eb8255b-f9fa-4570-a417-17c1845bcffc"). InnerVolumeSpecName "kube-api-access-ffr8r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:53:31.161513 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:31.161474    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0eb8255b-f9fa-4570-a417-17c1845bcffc-tls-certs\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:53:31.161513 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:31.161505    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0eb8255b-f9fa-4570-a417-17c1845bcffc-dshm\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:53:31.161513 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:31.161514    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffr8r\" (UniqueName: \"kubernetes.io/projected/0eb8255b-f9fa-4570-a417-17c1845bcffc-kube-api-access-ffr8r\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:53:31.306993 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:31.306901    2558 generic.go:356] "Generic (PLEG): container finished" podID="0eb8255b-f9fa-4570-a417-17c1845bcffc" containerID="adc75f1c8f7d5aa6cbd9db50e9dbdf180cde0bc6a9e246fc75b2a61a93fc6543" exitCode=0
Sep 12 00:53:31.306993 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:31.306948    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4"
Sep 12 00:53:31.307255 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:31.306947    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4" event={"ID":"0eb8255b-f9fa-4570-a417-17c1845bcffc","Type":"ContainerDied","Data":"adc75f1c8f7d5aa6cbd9db50e9dbdf180cde0bc6a9e246fc75b2a61a93fc6543"}
Sep 12 00:53:31.307255 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:31.307113    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4" event={"ID":"0eb8255b-f9fa-4570-a417-17c1845bcffc","Type":"ContainerDied","Data":"099932bac337a5b1d4c083a464a3c94cc910bb418da7ba43c5eed7171a550996"}
Sep 12 00:53:31.307255 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:31.307147    2558 scope.go:117] "RemoveContainer" containerID="adc75f1c8f7d5aa6cbd9db50e9dbdf180cde0bc6a9e246fc75b2a61a93fc6543"
Sep 12 00:53:31.316560 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:31.316542    2558 scope.go:117] "RemoveContainer" containerID="adc75f1c8f7d5aa6cbd9db50e9dbdf180cde0bc6a9e246fc75b2a61a93fc6543"
Sep 12 00:53:31.316890 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:53:31.316872    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adc75f1c8f7d5aa6cbd9db50e9dbdf180cde0bc6a9e246fc75b2a61a93fc6543\": container with ID starting with adc75f1c8f7d5aa6cbd9db50e9dbdf180cde0bc6a9e246fc75b2a61a93fc6543 not found: ID does not exist" containerID="adc75f1c8f7d5aa6cbd9db50e9dbdf180cde0bc6a9e246fc75b2a61a93fc6543"
Sep 12 00:53:31.316944 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:31.316900    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adc75f1c8f7d5aa6cbd9db50e9dbdf180cde0bc6a9e246fc75b2a61a93fc6543"} err="failed to get container status \"adc75f1c8f7d5aa6cbd9db50e9dbdf180cde0bc6a9e246fc75b2a61a93fc6543\": rpc error: code = NotFound desc = could not find container \"adc75f1c8f7d5aa6cbd9db50e9dbdf180cde0bc6a9e246fc75b2a61a93fc6543\": container with ID starting with adc75f1c8f7d5aa6cbd9db50e9dbdf180cde0bc6a9e246fc75b2a61a93fc6543 not found: ID does not exist"
Sep 12 00:53:31.328779 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:31.328751    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4"]
Sep 12 00:53:31.333317 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:31.333293    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-904cd0/noten-model-904cd0-kserve-9fb476dcf-q7zl4"]
Sep 12 00:53:31.743635 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:31.743590    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0eb8255b-f9fa-4570-a417-17c1845bcffc" path="/var/lib/kubelet/pods/0eb8255b-f9fa-4570-a417-17c1845bcffc/volumes"
Sep 12 00:53:31.744016 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:31.744002    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81b591cf-5ed8-4327-91a0-52e95b76c971" path="/var/lib/kubelet/pods/81b591cf-5ed8-4327-91a0-52e95b76c971/volumes"
Sep 12 00:53:42.254714 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.254671    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s"]
Sep 12 00:53:42.255155 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.255139    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="975e8170-7e78-40dd-a439-a24bd4a50165" containerName="payload-processing"
Sep 12 00:53:42.255223 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.255158    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="975e8170-7e78-40dd-a439-a24bd4a50165" containerName="payload-processing"
Sep 12 00:53:42.255223 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.255168    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0eb8255b-f9fa-4570-a417-17c1845bcffc" containerName="main"
Sep 12 00:53:42.255223 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.255174    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0eb8255b-f9fa-4570-a417-17c1845bcffc" containerName="main"
Sep 12 00:53:42.255223 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.255216    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e38f1710-8367-4e0d-a2d5-e5b277194c2a" containerName="payload-pre-processing"
Sep 12 00:53:42.255223 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.255224    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e38f1710-8367-4e0d-a2d5-e5b277194c2a" containerName="payload-pre-processing"
Sep 12 00:53:42.255417 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.255234    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81b591cf-5ed8-4327-91a0-52e95b76c971" containerName="main"
Sep 12 00:53:42.255417 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.255239    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="81b591cf-5ed8-4327-91a0-52e95b76c971" containerName="main"
Sep 12 00:53:42.255417 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.255303    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="975e8170-7e78-40dd-a439-a24bd4a50165" containerName="payload-processing"
Sep 12 00:53:42.255417 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.255315    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e38f1710-8367-4e0d-a2d5-e5b277194c2a" containerName="payload-pre-processing"
Sep 12 00:53:42.255417 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.255322    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0eb8255b-f9fa-4570-a417-17c1845bcffc" containerName="main"
Sep 12 00:53:42.255417 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.255330    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="81b591cf-5ed8-4327-91a0-52e95b76c971" containerName="main"
Sep 12 00:53:42.260129 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.260106    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s"
Sep 12 00:53:42.264196 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.264171    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-876808\"/\"kube-root-ca.crt\""
Sep 12 00:53:42.264325 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.264203    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-876808\"/\"openshift-service-ca.crt\""
Sep 12 00:53:42.264325 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.264308    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-876808\"/\"e2e-conflict-b-f937eb5b-kserve-self-signed-certs\""
Sep 12 00:53:42.264419 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.264350    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-876808\"/\"default-dockercfg-2c5sq\""
Sep 12 00:53:42.273321 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.273296    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s"]
Sep 12 00:53:42.291953 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.291919    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:42.375435 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.375402    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0e799dc-85cf-49b1-8486-c1d4f2978624-tmp-dir\") pod \"e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s\" (UID: \"f0e799dc-85cf-49b1-8486-c1d4f2978624\") " pod="e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s"
Sep 12 00:53:42.375633 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.375530    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0e799dc-85cf-49b1-8486-c1d4f2978624-home\") pod \"e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s\" (UID: \"f0e799dc-85cf-49b1-8486-c1d4f2978624\") " pod="e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s"
Sep 12 00:53:42.375633 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.375562    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0e799dc-85cf-49b1-8486-c1d4f2978624-tls-certs\") pod \"e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s\" (UID: \"f0e799dc-85cf-49b1-8486-c1d4f2978624\") " pod="e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s"
Sep 12 00:53:42.375633 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.375590    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0e799dc-85cf-49b1-8486-c1d4f2978624-dshm\") pod \"e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s\" (UID: \"f0e799dc-85cf-49b1-8486-c1d4f2978624\") " pod="e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s"
Sep 12 00:53:42.375633 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.375611    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0e799dc-85cf-49b1-8486-c1d4f2978624-model-cache\") pod \"e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s\" (UID: \"f0e799dc-85cf-49b1-8486-c1d4f2978624\") " pod="e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s"
Sep 12 00:53:42.375794 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.375641    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7xg4\" (UniqueName: \"kubernetes.io/projected/f0e799dc-85cf-49b1-8486-c1d4f2978624-kube-api-access-x7xg4\") pod \"e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s\" (UID: \"f0e799dc-85cf-49b1-8486-c1d4f2978624\") " pod="e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s"
Sep 12 00:53:42.476721 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.476677    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0e799dc-85cf-49b1-8486-c1d4f2978624-tmp-dir\") pod \"e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s\" (UID: \"f0e799dc-85cf-49b1-8486-c1d4f2978624\") " pod="e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s"
Sep 12 00:53:42.476935 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.476799    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0e799dc-85cf-49b1-8486-c1d4f2978624-home\") pod \"e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s\" (UID: \"f0e799dc-85cf-49b1-8486-c1d4f2978624\") " pod="e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s"
Sep 12 00:53:42.476935 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.476836    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0e799dc-85cf-49b1-8486-c1d4f2978624-tls-certs\") pod \"e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s\" (UID: \"f0e799dc-85cf-49b1-8486-c1d4f2978624\") " pod="e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s"
Sep 12 00:53:42.476935 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.476873    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0e799dc-85cf-49b1-8486-c1d4f2978624-dshm\") pod \"e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s\" (UID: \"f0e799dc-85cf-49b1-8486-c1d4f2978624\") " pod="e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s"
Sep 12 00:53:42.476935 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.476903    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0e799dc-85cf-49b1-8486-c1d4f2978624-model-cache\") pod \"e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s\" (UID: \"f0e799dc-85cf-49b1-8486-c1d4f2978624\") " pod="e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s"
Sep 12 00:53:42.477197 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.477026    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7xg4\" (UniqueName: \"kubernetes.io/projected/f0e799dc-85cf-49b1-8486-c1d4f2978624-kube-api-access-x7xg4\") pod \"e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s\" (UID: \"f0e799dc-85cf-49b1-8486-c1d4f2978624\") " pod="e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s"
Sep 12 00:53:42.477197 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.477073    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0e799dc-85cf-49b1-8486-c1d4f2978624-tmp-dir\") pod \"e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s\" (UID: \"f0e799dc-85cf-49b1-8486-c1d4f2978624\") " pod="e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s"
Sep 12 00:53:42.477305 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.477211    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0e799dc-85cf-49b1-8486-c1d4f2978624-home\") pod \"e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s\" (UID: \"f0e799dc-85cf-49b1-8486-c1d4f2978624\") " pod="e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s"
Sep 12 00:53:42.477404 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.477388    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0e799dc-85cf-49b1-8486-c1d4f2978624-model-cache\") pod \"e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s\" (UID: \"f0e799dc-85cf-49b1-8486-c1d4f2978624\") " pod="e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s"
Sep 12 00:53:42.479452 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.479428    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0e799dc-85cf-49b1-8486-c1d4f2978624-dshm\") pod \"e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s\" (UID: \"f0e799dc-85cf-49b1-8486-c1d4f2978624\") " pod="e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s"
Sep 12 00:53:42.479669 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.479653    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0e799dc-85cf-49b1-8486-c1d4f2978624-tls-certs\") pod \"e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s\" (UID: \"f0e799dc-85cf-49b1-8486-c1d4f2978624\") " pod="e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s"
Sep 12 00:53:42.486395 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.486367    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7xg4\" (UniqueName: \"kubernetes.io/projected/f0e799dc-85cf-49b1-8486-c1d4f2978624-kube-api-access-x7xg4\") pod \"e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s\" (UID: \"f0e799dc-85cf-49b1-8486-c1d4f2978624\") " pod="e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s"
Sep 12 00:53:42.572327 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.572240    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s"
Sep 12 00:53:42.847010 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:42.846983    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s"]
Sep 12 00:53:42.848877 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:53:42.848848    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf0e799dc_85cf_49b1_8486_c1d4f2978624.slice/crio-aa59dbf3d72fb77a05206ab2d88289359ffd6cf193caad27d3f3c6970ab8291b WatchSource:0}: Error finding container aa59dbf3d72fb77a05206ab2d88289359ffd6cf193caad27d3f3c6970ab8291b: Status 404 returned error can't find the container with id aa59dbf3d72fb77a05206ab2d88289359ffd6cf193caad27d3f3c6970ab8291b
Sep 12 00:53:43.371076 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:43.371042    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s" event={"ID":"f0e799dc-85cf-49b1-8486-c1d4f2978624","Type":"ContainerStarted","Data":"e4e85b833849c417ca1609521a815a2ea83fe7f7f657b55bd903ee67dc3240bb"}
Sep 12 00:53:43.371537 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:43.371114    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s" event={"ID":"f0e799dc-85cf-49b1-8486-c1d4f2978624","Type":"ContainerStarted","Data":"aa59dbf3d72fb77a05206ab2d88289359ffd6cf193caad27d3f3c6970ab8291b"}
Sep 12 00:53:43.387560 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:43.387509    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s" podStartSLOduration=1.387493488 podStartE2EDuration="1.387493488s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:53:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:53:43.386802361 +0000 UTC m=+1460.060861221" watchObservedRunningTime="2026-09-12 00:53:43.387493488 +0000 UTC m=+1460.061552344"
Sep 12 00:53:45.362464 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:45.362426    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:48.912726 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:48.912693    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j"]
Sep 12 00:53:48.921632 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:48.921605    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j"
Sep 12 00:53:48.925012 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:48.924983    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-5306e6ca\"/\"kube-root-ca.crt\""
Sep 12 00:53:48.925213 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:48.925171    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-5306e6ca\"/\"openshift-service-ca.crt\""
Sep 12 00:53:48.925999 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:48.925979    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-5306e6ca\"/\"e2e-lifecycle-model-5306e6ca-kserve-self-signed-certs\""
Sep 12 00:53:48.926136 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:48.925983    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-5306e6ca\"/\"default-dockercfg-qb7mv\""
Sep 12 00:53:48.926500 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:48.926483    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j"]
Sep 12 00:53:48.943598 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:48.943563    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:49.044463 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:49.044428    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9f622\" (UniqueName: \"kubernetes.io/projected/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-kube-api-access-9f622\") pod \"e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j\" (UID: \"38bf2c1a-1ee3-48a7-8273-a374436ccc1c\") " pod="ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j"
Sep 12 00:53:49.044623 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:49.044559    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-model-cache\") pod \"e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j\" (UID: \"38bf2c1a-1ee3-48a7-8273-a374436ccc1c\") " pod="ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j"
Sep 12 00:53:49.044623 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:49.044617    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-dshm\") pod \"e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j\" (UID: \"38bf2c1a-1ee3-48a7-8273-a374436ccc1c\") " pod="ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j"
Sep 12 00:53:49.044746 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:49.044642    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-home\") pod \"e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j\" (UID: \"38bf2c1a-1ee3-48a7-8273-a374436ccc1c\") " pod="ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j"
Sep 12 00:53:49.044746 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:49.044659    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-tmp-dir\") pod \"e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j\" (UID: \"38bf2c1a-1ee3-48a7-8273-a374436ccc1c\") " pod="ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j"
Sep 12 00:53:49.044847 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:49.044755    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-tls-certs\") pod \"e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j\" (UID: \"38bf2c1a-1ee3-48a7-8273-a374436ccc1c\") " pod="ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j"
Sep 12 00:53:49.146362 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:49.146317    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-model-cache\") pod \"e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j\" (UID: \"38bf2c1a-1ee3-48a7-8273-a374436ccc1c\") " pod="ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j"
Sep 12 00:53:49.146541 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:49.146387    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-dshm\") pod \"e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j\" (UID: \"38bf2c1a-1ee3-48a7-8273-a374436ccc1c\") " pod="ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j"
Sep 12 00:53:49.146541 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:49.146464    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-home\") pod \"e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j\" (UID: \"38bf2c1a-1ee3-48a7-8273-a374436ccc1c\") " pod="ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j"
Sep 12 00:53:49.146541 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:49.146506    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-tmp-dir\") pod \"e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j\" (UID: \"38bf2c1a-1ee3-48a7-8273-a374436ccc1c\") " pod="ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j"
Sep 12 00:53:49.146681 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:49.146554    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-tls-certs\") pod \"e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j\" (UID: \"38bf2c1a-1ee3-48a7-8273-a374436ccc1c\") " pod="ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j"
Sep 12 00:53:49.146681 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:49.146590    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9f622\" (UniqueName: \"kubernetes.io/projected/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-kube-api-access-9f622\") pod \"e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j\" (UID: \"38bf2c1a-1ee3-48a7-8273-a374436ccc1c\") " pod="ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j"
Sep 12 00:53:49.146812 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:49.146745    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-model-cache\") pod \"e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j\" (UID: \"38bf2c1a-1ee3-48a7-8273-a374436ccc1c\") " pod="ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j"
Sep 12 00:53:49.146865 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:49.146826    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-home\") pod \"e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j\" (UID: \"38bf2c1a-1ee3-48a7-8273-a374436ccc1c\") " pod="ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j"
Sep 12 00:53:49.146906 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:49.146882    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-tmp-dir\") pod \"e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j\" (UID: \"38bf2c1a-1ee3-48a7-8273-a374436ccc1c\") " pod="ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j"
Sep 12 00:53:49.148797 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:49.148779    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-dshm\") pod \"e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j\" (UID: \"38bf2c1a-1ee3-48a7-8273-a374436ccc1c\") " pod="ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j"
Sep 12 00:53:49.149130 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:49.149112    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-tls-certs\") pod \"e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j\" (UID: \"38bf2c1a-1ee3-48a7-8273-a374436ccc1c\") " pod="ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j"
Sep 12 00:53:49.162849 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:49.162784    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9f622\" (UniqueName: \"kubernetes.io/projected/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-kube-api-access-9f622\") pod \"e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j\" (UID: \"38bf2c1a-1ee3-48a7-8273-a374436ccc1c\") " pod="ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j"
Sep 12 00:53:49.234604 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:49.234567    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j"
Sep 12 00:53:49.510686 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:49.510657    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j"]
Sep 12 00:53:49.512905 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:53:49.512875    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod38bf2c1a_1ee3_48a7_8273_a374436ccc1c.slice/crio-22b519fce630e2bc36ab606d8f2c98d4f3638059f59518fbce2751d75bf56a0e WatchSource:0}: Error finding container 22b519fce630e2bc36ab606d8f2c98d4f3638059f59518fbce2751d75bf56a0e: Status 404 returned error can't find the container with id 22b519fce630e2bc36ab606d8f2c98d4f3638059f59518fbce2751d75bf56a0e
Sep 12 00:53:50.339723 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:50.339683    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:50.393946 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:50.393909    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:50.408880 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:50.408842    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j" event={"ID":"38bf2c1a-1ee3-48a7-8273-a374436ccc1c","Type":"ContainerStarted","Data":"e3cc5b779d1a091be1bf506cc92996b8296800c4ba7898b72c755a5d89eb6a1f"}
Sep 12 00:53:50.408880 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:50.408886    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j" event={"ID":"38bf2c1a-1ee3-48a7-8273-a374436ccc1c","Type":"ContainerStarted","Data":"22b519fce630e2bc36ab606d8f2c98d4f3638059f59518fbce2751d75bf56a0e"}
Sep 12 00:53:50.428014 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:50.427964    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j" podStartSLOduration=2.427944909 podStartE2EDuration="2.427944909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:53:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:53:50.426658496 +0000 UTC m=+1467.100717355" watchObservedRunningTime="2026-09-12 00:53:50.427944909 +0000 UTC m=+1467.102003769"
Sep 12 00:53:52.572545 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:52.572516    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s"
Sep 12 00:53:52.572930 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:52.572581    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s"
Sep 12 00:53:52.585604 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:52.585577    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s"
Sep 12 00:53:52.786623 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:52.786592    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:53.436254 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:53.436225    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s"
Sep 12 00:53:56.772434 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:56.772401    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:57.018543 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:57.018515    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:57.557000 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:57.556963    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:57.874296 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:57.874156    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:58.078207 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:58.078160    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:53:59.235565 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:59.235526    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j"
Sep 12 00:53:59.236045 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:59.235648    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j"
Sep 12 00:53:59.248647 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:59.248620    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j"
Sep 12 00:53:59.466836 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:59.466805    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j"
Sep 12 00:53:59.773394 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:53:59.773355    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:00.573951 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:00.573918    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:00.671226 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:00.671190    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:01.406989 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:01.406955    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:02.385273 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:02.385244    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:03.102926 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:03.102891    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:03.869013 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:03.868982    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:04.645227 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:04.645181    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:04.748291 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:04.748256    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:04.845875 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:04.845834    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:05.183792 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:05.183759    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:05.255039 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:05.255004    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-9511bd-69d77c4d7f-6qgz6"]
Sep 12 00:54:05.259070 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:05.259053    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-9511bd-69d77c4d7f-6qgz6"
Sep 12 00:54:05.261663 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:05.261640    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-9511bd-dockercfg-sfrp7\""
Sep 12 00:54:05.261812 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:05.261718    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-9511bd\""
Sep 12 00:54:05.270774 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:05.270750    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-9511bd-69d77c4d7f-6qgz6"]
Sep 12 00:54:05.272214 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:05.272188    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:05.297724 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:05.297691    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7311aa4-3f0a-45c1-a5ba-cf05195cfea8-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-9511bd-69d77c4d7f-6qgz6\" (UID: \"d7311aa4-3f0a-45c1-a5ba-cf05195cfea8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-9511bd-69d77c4d7f-6qgz6"
Sep 12 00:54:05.297872 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:05.297742    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5487g\" (UniqueName: \"kubernetes.io/projected/d7311aa4-3f0a-45c1-a5ba-cf05195cfea8-kube-api-access-5487g\") pod \"payload-processing-e2e-shared-a-w3-9511bd-69d77c4d7f-6qgz6\" (UID: \"d7311aa4-3f0a-45c1-a5ba-cf05195cfea8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-9511bd-69d77c4d7f-6qgz6"
Sep 12 00:54:05.322124 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:05.322072    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9511bd-8d458d974-85lsm"]
Sep 12 00:54:05.325778 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:05.325755    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9511bd-8d458d974-85lsm"
Sep 12 00:54:05.335843 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:05.335820    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9511bd-8d458d974-85lsm"]
Sep 12 00:54:05.398511 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:05.398478    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4eb4187-af72-4a2f-95de-3fd65ae9dd77-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-9511bd-8d458d974-85lsm\" (UID: \"d4eb4187-af72-4a2f-95de-3fd65ae9dd77\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9511bd-8d458d974-85lsm"
Sep 12 00:54:05.398700 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:05.398526    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7311aa4-3f0a-45c1-a5ba-cf05195cfea8-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-9511bd-69d77c4d7f-6qgz6\" (UID: \"d7311aa4-3f0a-45c1-a5ba-cf05195cfea8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-9511bd-69d77c4d7f-6qgz6"
Sep 12 00:54:05.398700 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:05.398549    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5487g\" (UniqueName: \"kubernetes.io/projected/d7311aa4-3f0a-45c1-a5ba-cf05195cfea8-kube-api-access-5487g\") pod \"payload-processing-e2e-shared-a-w3-9511bd-69d77c4d7f-6qgz6\" (UID: \"d7311aa4-3f0a-45c1-a5ba-cf05195cfea8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-9511bd-69d77c4d7f-6qgz6"
Sep 12 00:54:05.398792 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:05.398709    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djbv6\" (UniqueName: \"kubernetes.io/projected/d4eb4187-af72-4a2f-95de-3fd65ae9dd77-kube-api-access-djbv6\") pod \"payload-pre-processing-e2e-shared-a-w3-9511bd-8d458d974-85lsm\" (UID: \"d4eb4187-af72-4a2f-95de-3fd65ae9dd77\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9511bd-8d458d974-85lsm"
Sep 12 00:54:05.399271 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:05.399255    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7311aa4-3f0a-45c1-a5ba-cf05195cfea8-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-9511bd-69d77c4d7f-6qgz6\" (UID: \"d7311aa4-3f0a-45c1-a5ba-cf05195cfea8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-9511bd-69d77c4d7f-6qgz6"
Sep 12 00:54:05.406724 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:05.406697    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5487g\" (UniqueName: \"kubernetes.io/projected/d7311aa4-3f0a-45c1-a5ba-cf05195cfea8-kube-api-access-5487g\") pod \"payload-processing-e2e-shared-a-w3-9511bd-69d77c4d7f-6qgz6\" (UID: \"d7311aa4-3f0a-45c1-a5ba-cf05195cfea8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-9511bd-69d77c4d7f-6qgz6"
Sep 12 00:54:05.499561 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:05.499522    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4eb4187-af72-4a2f-95de-3fd65ae9dd77-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-9511bd-8d458d974-85lsm\" (UID: \"d4eb4187-af72-4a2f-95de-3fd65ae9dd77\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9511bd-8d458d974-85lsm"
Sep 12 00:54:05.499765 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:05.499666    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djbv6\" (UniqueName: \"kubernetes.io/projected/d4eb4187-af72-4a2f-95de-3fd65ae9dd77-kube-api-access-djbv6\") pod \"payload-pre-processing-e2e-shared-a-w3-9511bd-8d458d974-85lsm\" (UID: \"d4eb4187-af72-4a2f-95de-3fd65ae9dd77\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9511bd-8d458d974-85lsm"
Sep 12 00:54:05.500290 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:05.500269    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4eb4187-af72-4a2f-95de-3fd65ae9dd77-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-9511bd-8d458d974-85lsm\" (UID: \"d4eb4187-af72-4a2f-95de-3fd65ae9dd77\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9511bd-8d458d974-85lsm"
Sep 12 00:54:05.507495 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:05.507472    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djbv6\" (UniqueName: \"kubernetes.io/projected/d4eb4187-af72-4a2f-95de-3fd65ae9dd77-kube-api-access-djbv6\") pod \"payload-pre-processing-e2e-shared-a-w3-9511bd-8d458d974-85lsm\" (UID: \"d4eb4187-af72-4a2f-95de-3fd65ae9dd77\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9511bd-8d458d974-85lsm"
Sep 12 00:54:05.571671 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:05.571630    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-9511bd-69d77c4d7f-6qgz6"
Sep 12 00:54:05.637245 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:05.636897    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9511bd-8d458d974-85lsm"
Sep 12 00:54:05.845321 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:05.845288    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-9511bd-69d77c4d7f-6qgz6"]
Sep 12 00:54:05.847364 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:54:05.847328    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7311aa4_3f0a_45c1_a5ba_cf05195cfea8.slice/crio-fcdf2a23110844404510a471fd8b77a02777cebe87d90044719b0a81f32cb242 WatchSource:0}: Error finding container fcdf2a23110844404510a471fd8b77a02777cebe87d90044719b0a81f32cb242: Status 404 returned error can't find the container with id fcdf2a23110844404510a471fd8b77a02777cebe87d90044719b0a81f32cb242
Sep 12 00:54:05.915909 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:05.915885    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9511bd-8d458d974-85lsm"]
Sep 12 00:54:05.917696 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:54:05.917655    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4eb4187_af72_4a2f_95de_3fd65ae9dd77.slice/crio-d8b2bd5d23ae36e0ed1ee4e1fe9291e7341b64ba74f81c3beab54a89c6b8c65b WatchSource:0}: Error finding container d8b2bd5d23ae36e0ed1ee4e1fe9291e7341b64ba74f81c3beab54a89c6b8c65b: Status 404 returned error can't find the container with id d8b2bd5d23ae36e0ed1ee4e1fe9291e7341b64ba74f81c3beab54a89c6b8c65b
Sep 12 00:54:06.496604 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:06.496557    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9511bd-8d458d974-85lsm" event={"ID":"d4eb4187-af72-4a2f-95de-3fd65ae9dd77","Type":"ContainerStarted","Data":"0bd73918029f9dba8030a8678e3000e7ff5f0bc2b69b814a9a7d527a436655d5"}
Sep 12 00:54:06.497139 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:06.496622    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9511bd-8d458d974-85lsm" event={"ID":"d4eb4187-af72-4a2f-95de-3fd65ae9dd77","Type":"ContainerStarted","Data":"d8b2bd5d23ae36e0ed1ee4e1fe9291e7341b64ba74f81c3beab54a89c6b8c65b"}
Sep 12 00:54:06.497139 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:06.496731    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9511bd-8d458d974-85lsm"
Sep 12 00:54:06.498637 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:06.498615    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-9511bd-69d77c4d7f-6qgz6" event={"ID":"d7311aa4-3f0a-45c1-a5ba-cf05195cfea8","Type":"ContainerStarted","Data":"056de2fb8a17cb63ed3b3de44769dcf9182e36387611265bb3987c92430d9484"}
Sep 12 00:54:06.498786 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:06.498661    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-9511bd-69d77c4d7f-6qgz6" event={"ID":"d7311aa4-3f0a-45c1-a5ba-cf05195cfea8","Type":"ContainerStarted","Data":"fcdf2a23110844404510a471fd8b77a02777cebe87d90044719b0a81f32cb242"}
Sep 12 00:54:06.498786 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:06.498698    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-9511bd-69d77c4d7f-6qgz6"
Sep 12 00:54:06.514012 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:06.513955    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9511bd-8d458d974-85lsm" podStartSLOduration=1.513937597 podStartE2EDuration="1.513937597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:54:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:54:06.511926933 +0000 UTC m=+1483.185985804" watchObservedRunningTime="2026-09-12 00:54:06.513937597 +0000 UTC m=+1483.187996455"
Sep 12 00:54:06.529804 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:06.529754    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-9511bd-69d77c4d7f-6qgz6" podStartSLOduration=1.529738641 podStartE2EDuration="1.529738641s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:54:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:54:06.527286776 +0000 UTC m=+1483.201345637" watchObservedRunningTime="2026-09-12 00:54:06.529738641 +0000 UTC m=+1483.203797500"
Sep 12 00:54:17.505209 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:17.505163    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9511bd-8d458d974-85lsm"
Sep 12 00:54:17.505654 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:17.505256    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-9511bd-69d77c4d7f-6qgz6"
Sep 12 00:54:23.828814 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:23.828709    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-czf8h_dd90996a-5428-4ccf-a154-c0aa8def9789/console-operator/1.log"
Sep 12 00:54:23.841910 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:23.831861    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-czf8h_dd90996a-5428-4ccf-a154-c0aa8def9789/console-operator/1.log"
Sep 12 00:54:26.676380 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:26.676336    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:27.411971 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:27.411939    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:28.488452 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:28.488396    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:28.981370 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:28.981332    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:29.642948 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:29.642912    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:30.292999 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:30.292961    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:30.511649 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:30.511609    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:30.847157 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:30.847116    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:31.096423 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:31.096389    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:31.194984 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:31.194930    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:31.381159 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:31.381127    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:33.967437 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:33.967398    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:34.676962 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:34.676930    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:34.776665 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:34.776632    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:35.432095 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:35.432049    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:36.241043 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:36.241005    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:36.986557 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:36.986523    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:37.663213 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:37.663176    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:38.371403 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:38.371368    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:39.116258 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:39.116226    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:39.214620 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:39.214587    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:39.578953 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:39.578918    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:39.679127 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:39.679073    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:39.681515 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:39.681490    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-dd25dd-85477c9b8b-92knm"]
Sep 12 00:54:39.686610 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:39.686587    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-dd25dd-85477c9b8b-92knm"
Sep 12 00:54:39.689536 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:39.689517    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-dd25dd\""
Sep 12 00:54:39.697583 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:39.697556    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-dd25dd-85477c9b8b-92knm"]
Sep 12 00:54:39.741269 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:39.741235    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfg92\" (UniqueName: \"kubernetes.io/projected/e2ef4545-d14b-4a5c-a2af-80962ed7b545-kube-api-access-cfg92\") pod \"payload-pre-processing-e2e-shared-b-w3-dd25dd-85477c9b8b-92knm\" (UID: \"e2ef4545-d14b-4a5c-a2af-80962ed7b545\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-dd25dd-85477c9b8b-92knm"
Sep 12 00:54:39.741432 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:39.741391    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2ef4545-d14b-4a5c-a2af-80962ed7b545-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-dd25dd-85477c9b8b-92knm\" (UID: \"e2ef4545-d14b-4a5c-a2af-80962ed7b545\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-dd25dd-85477c9b8b-92knm"
Sep 12 00:54:39.770558 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:39.770524    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:39.843064 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:39.842973    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfg92\" (UniqueName: \"kubernetes.io/projected/e2ef4545-d14b-4a5c-a2af-80962ed7b545-kube-api-access-cfg92\") pod \"payload-pre-processing-e2e-shared-b-w3-dd25dd-85477c9b8b-92knm\" (UID: \"e2ef4545-d14b-4a5c-a2af-80962ed7b545\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-dd25dd-85477c9b8b-92knm"
Sep 12 00:54:39.843264 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:39.843065    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2ef4545-d14b-4a5c-a2af-80962ed7b545-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-dd25dd-85477c9b8b-92knm\" (UID: \"e2ef4545-d14b-4a5c-a2af-80962ed7b545\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-dd25dd-85477c9b8b-92knm"
Sep 12 00:54:39.844003 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:39.843975    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2ef4545-d14b-4a5c-a2af-80962ed7b545-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-dd25dd-85477c9b8b-92knm\" (UID: \"e2ef4545-d14b-4a5c-a2af-80962ed7b545\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-dd25dd-85477c9b8b-92knm"
Sep 12 00:54:39.851024 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:39.850997    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfg92\" (UniqueName: \"kubernetes.io/projected/e2ef4545-d14b-4a5c-a2af-80962ed7b545-kube-api-access-cfg92\") pod \"payload-pre-processing-e2e-shared-b-w3-dd25dd-85477c9b8b-92knm\" (UID: \"e2ef4545-d14b-4a5c-a2af-80962ed7b545\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-dd25dd-85477c9b8b-92knm"
Sep 12 00:54:39.997781 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:39.997747    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-dd25dd-85477c9b8b-92knm"
Sep 12 00:54:40.273678 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:40.273646    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-dd25dd-85477c9b8b-92knm"]
Sep 12 00:54:40.275148 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:54:40.275116    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2ef4545_d14b_4a5c_a2af_80962ed7b545.slice/crio-749ebcc0d739b76b08db51d927e6388c7d3db0ee38bd5df81110e7e52c082031 WatchSource:0}: Error finding container 749ebcc0d739b76b08db51d927e6388c7d3db0ee38bd5df81110e7e52c082031: Status 404 returned error can't find the container with id 749ebcc0d739b76b08db51d927e6388c7d3db0ee38bd5df81110e7e52c082031
Sep 12 00:54:40.684130 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:40.684023    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-dd25dd-85477c9b8b-92knm" event={"ID":"e2ef4545-d14b-4a5c-a2af-80962ed7b545","Type":"ContainerStarted","Data":"e18412f50547dbaf98df0474d689d49063ae169c8a479ab0c1f99ef110f87660"}
Sep 12 00:54:40.684130 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:40.684079    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-dd25dd-85477c9b8b-92knm" event={"ID":"e2ef4545-d14b-4a5c-a2af-80962ed7b545","Type":"ContainerStarted","Data":"749ebcc0d739b76b08db51d927e6388c7d3db0ee38bd5df81110e7e52c082031"}
Sep 12 00:54:40.684572 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:40.684244    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-dd25dd-85477c9b8b-92knm"
Sep 12 00:54:40.700857 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:40.700806    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-dd25dd-85477c9b8b-92knm" podStartSLOduration=1.7007908390000002 podStartE2EDuration="1.700790839s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:54:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:54:40.698770387 +0000 UTC m=+1517.372829246" watchObservedRunningTime="2026-09-12 00:54:40.700790839 +0000 UTC m=+1517.374849698"
Sep 12 00:54:50.306783 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:50.306696    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw"]
Sep 12 00:54:50.311315 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:50.311290    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw"
Sep 12 00:54:50.313848 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:50.313824    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-4d485e\"/\"distinct-2-w0-4d485e-kserve-self-signed-certs\""
Sep 12 00:54:50.319931 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:50.319887    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw"]
Sep 12 00:54:50.353810 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:50.353777    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-tls-certs\") pod \"distinct-2-w0-4d485e-kserve-544cc565c4-gknqw\" (UID: \"a19c70cb-f6f2-4f28-b05b-eeeda341a3c8\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw"
Sep 12 00:54:50.354000 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:50.353837    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-model-cache\") pod \"distinct-2-w0-4d485e-kserve-544cc565c4-gknqw\" (UID: \"a19c70cb-f6f2-4f28-b05b-eeeda341a3c8\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw"
Sep 12 00:54:50.354000 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:50.353879    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-home\") pod \"distinct-2-w0-4d485e-kserve-544cc565c4-gknqw\" (UID: \"a19c70cb-f6f2-4f28-b05b-eeeda341a3c8\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw"
Sep 12 00:54:50.354000 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:50.353925    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2497j\" (UniqueName: \"kubernetes.io/projected/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-kube-api-access-2497j\") pod \"distinct-2-w0-4d485e-kserve-544cc565c4-gknqw\" (UID: \"a19c70cb-f6f2-4f28-b05b-eeeda341a3c8\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw"
Sep 12 00:54:50.354000 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:50.353946    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-tmp-dir\") pod \"distinct-2-w0-4d485e-kserve-544cc565c4-gknqw\" (UID: \"a19c70cb-f6f2-4f28-b05b-eeeda341a3c8\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw"
Sep 12 00:54:50.354000 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:50.353978    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-dshm\") pod \"distinct-2-w0-4d485e-kserve-544cc565c4-gknqw\" (UID: \"a19c70cb-f6f2-4f28-b05b-eeeda341a3c8\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw"
Sep 12 00:54:50.420636 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:50.420598    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:54:50.455488 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:50.455448    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-model-cache\") pod \"distinct-2-w0-4d485e-kserve-544cc565c4-gknqw\" (UID: \"a19c70cb-f6f2-4f28-b05b-eeeda341a3c8\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw"
Sep 12 00:54:50.455692 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:50.455500    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-home\") pod \"distinct-2-w0-4d485e-kserve-544cc565c4-gknqw\" (UID: \"a19c70cb-f6f2-4f28-b05b-eeeda341a3c8\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw"
Sep 12 00:54:50.455692 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:50.455547    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2497j\" (UniqueName: \"kubernetes.io/projected/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-kube-api-access-2497j\") pod \"distinct-2-w0-4d485e-kserve-544cc565c4-gknqw\" (UID: \"a19c70cb-f6f2-4f28-b05b-eeeda341a3c8\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw"
Sep 12 00:54:50.455692 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:50.455574    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-tmp-dir\") pod \"distinct-2-w0-4d485e-kserve-544cc565c4-gknqw\" (UID: \"a19c70cb-f6f2-4f28-b05b-eeeda341a3c8\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw"
Sep 12 00:54:50.455692 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:50.455634    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-dshm\") pod \"distinct-2-w0-4d485e-kserve-544cc565c4-gknqw\" (UID: \"a19c70cb-f6f2-4f28-b05b-eeeda341a3c8\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw"
Sep 12 00:54:50.455938 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:50.455713    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-tls-certs\") pod \"distinct-2-w0-4d485e-kserve-544cc565c4-gknqw\" (UID: \"a19c70cb-f6f2-4f28-b05b-eeeda341a3c8\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw"
Sep 12 00:54:50.455938 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:50.455908    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-model-cache\") pod \"distinct-2-w0-4d485e-kserve-544cc565c4-gknqw\" (UID: \"a19c70cb-f6f2-4f28-b05b-eeeda341a3c8\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw"
Sep 12 00:54:50.455938 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:50.455929    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-home\") pod \"distinct-2-w0-4d485e-kserve-544cc565c4-gknqw\" (UID: \"a19c70cb-f6f2-4f28-b05b-eeeda341a3c8\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw"
Sep 12 00:54:50.456132 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:50.455992    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-tmp-dir\") pod \"distinct-2-w0-4d485e-kserve-544cc565c4-gknqw\" (UID: \"a19c70cb-f6f2-4f28-b05b-eeeda341a3c8\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw"
Sep 12 00:54:50.458187 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:50.458166    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-dshm\") pod \"distinct-2-w0-4d485e-kserve-544cc565c4-gknqw\" (UID: \"a19c70cb-f6f2-4f28-b05b-eeeda341a3c8\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw"
Sep 12 00:54:50.458363 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:50.458347    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-tls-certs\") pod \"distinct-2-w0-4d485e-kserve-544cc565c4-gknqw\" (UID: \"a19c70cb-f6f2-4f28-b05b-eeeda341a3c8\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw"
Sep 12 00:54:50.463783 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:50.463764    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2497j\" (UniqueName: \"kubernetes.io/projected/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-kube-api-access-2497j\") pod \"distinct-2-w0-4d485e-kserve-544cc565c4-gknqw\" (UID: \"a19c70cb-f6f2-4f28-b05b-eeeda341a3c8\") " pod="e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw"
Sep 12 00:54:50.622352 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:50.622259    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw"
Sep 12 00:54:50.897494 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:50.897464    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw"]
Sep 12 00:54:50.899340 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:54:50.899312    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda19c70cb_f6f2_4f28_b05b_eeeda341a3c8.slice/crio-1edccae8600b819264ecbb83cdfcfcca224bce84818d7cb145e8f67c1a00e7d1 WatchSource:0}: Error finding container 1edccae8600b819264ecbb83cdfcfcca224bce84818d7cb145e8f67c1a00e7d1: Status 404 returned error can't find the container with id 1edccae8600b819264ecbb83cdfcfcca224bce84818d7cb145e8f67c1a00e7d1
Sep 12 00:54:51.693338 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:51.693305    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-dd25dd-85477c9b8b-92knm"
Sep 12 00:54:51.746960 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:51.746924    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw" event={"ID":"a19c70cb-f6f2-4f28-b05b-eeeda341a3c8","Type":"ContainerStarted","Data":"ebf24c63459e63436b4e8d1c952dc513cea20b5eb74c63172cd328a79d054d89"}
Sep 12 00:54:51.747192 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:51.746976    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw" event={"ID":"a19c70cb-f6f2-4f28-b05b-eeeda341a3c8","Type":"ContainerStarted","Data":"1edccae8600b819264ecbb83cdfcfcca224bce84818d7cb145e8f67c1a00e7d1"}
Sep 12 00:54:51.767318 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:51.767257    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw" podStartSLOduration=1.7672350369999998 podStartE2EDuration="1.767235037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:54:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:54:51.765709751 +0000 UTC m=+1528.439768610" watchObservedRunningTime="2026-09-12 00:54:51.767235037 +0000 UTC m=+1528.441293896"
Sep 12 00:54:57.041350 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:54:57.041300    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:00.622639 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:00.622603    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw"
Sep 12 00:55:00.623147 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:00.622688    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw"
Sep 12 00:55:00.635603 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:00.635576    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw"
Sep 12 00:55:00.805830 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:00.805800    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw"
Sep 12 00:55:00.986340 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:00.986307    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:01.979319 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:01.979286    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:02.767162 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:02.767132    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:03.480354 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:03.480319    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:04.288478 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:04.288443    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:04.728676 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:04.728645    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:04.987423 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:04.987331    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:05.681329 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:05.681267    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:06.466530 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:06.466479    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:21.702078 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:21.702042    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s"]
Sep 12 00:55:21.722716 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:21.722671    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:27.900376 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:27.900340    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:28.712293 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:28.712254    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf"]
Sep 12 00:55:28.717070 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:28.717041    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf"
Sep 12 00:55:28.719700 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:28.719675    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-9511bd\"/\"default-dockercfg-scc8f\""
Sep 12 00:55:28.719858 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:28.719675    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-9511bd\"/\"openshift-service-ca.crt\""
Sep 12 00:55:28.720763 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:28.720744    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-9511bd\"/\"auth-test-model-9511bd-kserve-self-signed-certs\""
Sep 12 00:55:28.720910 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:28.720888    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-9511bd\"/\"kube-root-ca.crt\""
Sep 12 00:55:28.725888 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:28.725839    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf"]
Sep 12 00:55:28.744359 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:28.744315    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:28.861142 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:28.861104    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmtc9\" (UniqueName: \"kubernetes.io/projected/62e1050f-0f91-41e1-af0c-5d3b3d73f515-kube-api-access-dmtc9\") pod \"auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf\" (UID: \"62e1050f-0f91-41e1-af0c-5d3b3d73f515\") " pod="ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf"
Sep 12 00:55:28.861142 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:28.861143    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62e1050f-0f91-41e1-af0c-5d3b3d73f515-home\") pod \"auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf\" (UID: \"62e1050f-0f91-41e1-af0c-5d3b3d73f515\") " pod="ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf"
Sep 12 00:55:28.861386 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:28.861166    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62e1050f-0f91-41e1-af0c-5d3b3d73f515-tmp-dir\") pod \"auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf\" (UID: \"62e1050f-0f91-41e1-af0c-5d3b3d73f515\") " pod="ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf"
Sep 12 00:55:28.861386 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:28.861322    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62e1050f-0f91-41e1-af0c-5d3b3d73f515-model-cache\") pod \"auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf\" (UID: \"62e1050f-0f91-41e1-af0c-5d3b3d73f515\") " pod="ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf"
Sep 12 00:55:28.861386 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:28.861365    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62e1050f-0f91-41e1-af0c-5d3b3d73f515-dshm\") pod \"auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf\" (UID: \"62e1050f-0f91-41e1-af0c-5d3b3d73f515\") " pod="ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf"
Sep 12 00:55:28.861546 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:28.861448    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62e1050f-0f91-41e1-af0c-5d3b3d73f515-tls-certs\") pod \"auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf\" (UID: \"62e1050f-0f91-41e1-af0c-5d3b3d73f515\") " pod="ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf"
Sep 12 00:55:28.962924 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:28.962838    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62e1050f-0f91-41e1-af0c-5d3b3d73f515-dshm\") pod \"auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf\" (UID: \"62e1050f-0f91-41e1-af0c-5d3b3d73f515\") " pod="ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf"
Sep 12 00:55:28.962924 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:28.962888    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62e1050f-0f91-41e1-af0c-5d3b3d73f515-tls-certs\") pod \"auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf\" (UID: \"62e1050f-0f91-41e1-af0c-5d3b3d73f515\") " pod="ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf"
Sep 12 00:55:28.963467 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:28.962943    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmtc9\" (UniqueName: \"kubernetes.io/projected/62e1050f-0f91-41e1-af0c-5d3b3d73f515-kube-api-access-dmtc9\") pod \"auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf\" (UID: \"62e1050f-0f91-41e1-af0c-5d3b3d73f515\") " pod="ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf"
Sep 12 00:55:28.963467 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:28.962961    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62e1050f-0f91-41e1-af0c-5d3b3d73f515-home\") pod \"auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf\" (UID: \"62e1050f-0f91-41e1-af0c-5d3b3d73f515\") " pod="ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf"
Sep 12 00:55:28.963467 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:28.962981    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62e1050f-0f91-41e1-af0c-5d3b3d73f515-tmp-dir\") pod \"auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf\" (UID: \"62e1050f-0f91-41e1-af0c-5d3b3d73f515\") " pod="ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf"
Sep 12 00:55:28.963467 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:28.963043    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62e1050f-0f91-41e1-af0c-5d3b3d73f515-model-cache\") pod \"auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf\" (UID: \"62e1050f-0f91-41e1-af0c-5d3b3d73f515\") " pod="ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf"
Sep 12 00:55:28.963467 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:28.963401    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62e1050f-0f91-41e1-af0c-5d3b3d73f515-home\") pod \"auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf\" (UID: \"62e1050f-0f91-41e1-af0c-5d3b3d73f515\") " pod="ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf"
Sep 12 00:55:28.963467 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:28.963428    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62e1050f-0f91-41e1-af0c-5d3b3d73f515-tmp-dir\") pod \"auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf\" (UID: \"62e1050f-0f91-41e1-af0c-5d3b3d73f515\") " pod="ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf"
Sep 12 00:55:28.963755 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:28.963482    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62e1050f-0f91-41e1-af0c-5d3b3d73f515-model-cache\") pod \"auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf\" (UID: \"62e1050f-0f91-41e1-af0c-5d3b3d73f515\") " pod="ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf"
Sep 12 00:55:28.965339 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:28.965320    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62e1050f-0f91-41e1-af0c-5d3b3d73f515-dshm\") pod \"auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf\" (UID: \"62e1050f-0f91-41e1-af0c-5d3b3d73f515\") " pod="ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf"
Sep 12 00:55:28.965642 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:28.965627    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62e1050f-0f91-41e1-af0c-5d3b3d73f515-tls-certs\") pod \"auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf\" (UID: \"62e1050f-0f91-41e1-af0c-5d3b3d73f515\") " pod="ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf"
Sep 12 00:55:28.972407 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:28.972386    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmtc9\" (UniqueName: \"kubernetes.io/projected/62e1050f-0f91-41e1-af0c-5d3b3d73f515-kube-api-access-dmtc9\") pod \"auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf\" (UID: \"62e1050f-0f91-41e1-af0c-5d3b3d73f515\") " pod="ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf"
Sep 12 00:55:29.030043 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:29.030010    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf"
Sep 12 00:55:29.305317 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:29.305288    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf"]
Sep 12 00:55:29.307477 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:55:29.307446    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod62e1050f_0f91_41e1_af0c_5d3b3d73f515.slice/crio-14cb6454c5a50d1fd0e6a099cc8c3691df27021a5405ed393b04c304715782ec WatchSource:0}: Error finding container 14cb6454c5a50d1fd0e6a099cc8c3691df27021a5405ed393b04c304715782ec: Status 404 returned error can't find the container with id 14cb6454c5a50d1fd0e6a099cc8c3691df27021a5405ed393b04c304715782ec
Sep 12 00:55:29.955555 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:29.955519    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf" event={"ID":"62e1050f-0f91-41e1-af0c-5d3b3d73f515","Type":"ContainerStarted","Data":"c8aef4659e957e2777466dbce246af54ea0501dd81aa7c25ff09cf2680ef1264"}
Sep 12 00:55:29.955555 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:29.955568    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf" event={"ID":"62e1050f-0f91-41e1-af0c-5d3b3d73f515","Type":"ContainerStarted","Data":"14cb6454c5a50d1fd0e6a099cc8c3691df27021a5405ed393b04c304715782ec"}
Sep 12 00:55:29.974996 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:29.974946    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf" podStartSLOduration=1.974927922 podStartE2EDuration="1.974927922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:55:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:55:29.971530628 +0000 UTC m=+1566.645589490" watchObservedRunningTime="2026-09-12 00:55:29.974927922 +0000 UTC m=+1566.648986781"
Sep 12 00:55:32.822377 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:32.822344    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:34.014903 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:34.014864    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:34.936431 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:34.936399    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:35.791244 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:35.791210    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:36.478313 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:36.478279    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:36.723898 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:36.723841    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s" podUID="f0e799dc-85cf-49b1-8486-c1d4f2978624" containerName="main" containerID="cri-o://e4e85b833849c417ca1609521a815a2ea83fe7f7f657b55bd903ee67dc3240bb" gracePeriod=45
Sep 12 00:55:36.984792 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:36.984767    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s"
Sep 12 00:55:37.000005 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:36.999927    2558 generic.go:356] "Generic (PLEG): container finished" podID="f0e799dc-85cf-49b1-8486-c1d4f2978624" containerID="e4e85b833849c417ca1609521a815a2ea83fe7f7f657b55bd903ee67dc3240bb" exitCode=0
Sep 12 00:55:37.000005 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:36.999970    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s"
Sep 12 00:55:37.000211 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:37.000016    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s" event={"ID":"f0e799dc-85cf-49b1-8486-c1d4f2978624","Type":"ContainerDied","Data":"e4e85b833849c417ca1609521a815a2ea83fe7f7f657b55bd903ee67dc3240bb"}
Sep 12 00:55:37.000211 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:37.000065    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s" event={"ID":"f0e799dc-85cf-49b1-8486-c1d4f2978624","Type":"ContainerDied","Data":"aa59dbf3d72fb77a05206ab2d88289359ffd6cf193caad27d3f3c6970ab8291b"}
Sep 12 00:55:37.000211 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:37.000114    2558 scope.go:117] "RemoveContainer" containerID="e4e85b833849c417ca1609521a815a2ea83fe7f7f657b55bd903ee67dc3240bb"
Sep 12 00:55:37.010275 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:37.010248    2558 scope.go:117] "RemoveContainer" containerID="e4e85b833849c417ca1609521a815a2ea83fe7f7f657b55bd903ee67dc3240bb"
Sep 12 00:55:37.010637 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:55:37.010614    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4e85b833849c417ca1609521a815a2ea83fe7f7f657b55bd903ee67dc3240bb\": container with ID starting with e4e85b833849c417ca1609521a815a2ea83fe7f7f657b55bd903ee67dc3240bb not found: ID does not exist" containerID="e4e85b833849c417ca1609521a815a2ea83fe7f7f657b55bd903ee67dc3240bb"
Sep 12 00:55:37.010747 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:37.010647    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4e85b833849c417ca1609521a815a2ea83fe7f7f657b55bd903ee67dc3240bb"} err="failed to get container status \"e4e85b833849c417ca1609521a815a2ea83fe7f7f657b55bd903ee67dc3240bb\": rpc error: code = NotFound desc = could not find container \"e4e85b833849c417ca1609521a815a2ea83fe7f7f657b55bd903ee67dc3240bb\": container with ID starting with e4e85b833849c417ca1609521a815a2ea83fe7f7f657b55bd903ee67dc3240bb not found: ID does not exist"
Sep 12 00:55:37.156645 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:37.156616    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0e799dc-85cf-49b1-8486-c1d4f2978624-tls-certs\") pod \"f0e799dc-85cf-49b1-8486-c1d4f2978624\" (UID: \"f0e799dc-85cf-49b1-8486-c1d4f2978624\") "
Sep 12 00:55:37.156645 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:37.156650    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0e799dc-85cf-49b1-8486-c1d4f2978624-dshm\") pod \"f0e799dc-85cf-49b1-8486-c1d4f2978624\" (UID: \"f0e799dc-85cf-49b1-8486-c1d4f2978624\") "
Sep 12 00:55:37.156904 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:37.156749    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0e799dc-85cf-49b1-8486-c1d4f2978624-model-cache\") pod \"f0e799dc-85cf-49b1-8486-c1d4f2978624\" (UID: \"f0e799dc-85cf-49b1-8486-c1d4f2978624\") "
Sep 12 00:55:37.156904 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:37.156780    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0e799dc-85cf-49b1-8486-c1d4f2978624-tmp-dir\") pod \"f0e799dc-85cf-49b1-8486-c1d4f2978624\" (UID: \"f0e799dc-85cf-49b1-8486-c1d4f2978624\") "
Sep 12 00:55:37.156904 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:37.156897    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7xg4\" (UniqueName: \"kubernetes.io/projected/f0e799dc-85cf-49b1-8486-c1d4f2978624-kube-api-access-x7xg4\") pod \"f0e799dc-85cf-49b1-8486-c1d4f2978624\" (UID: \"f0e799dc-85cf-49b1-8486-c1d4f2978624\") "
Sep 12 00:55:37.157076 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:37.156945    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0e799dc-85cf-49b1-8486-c1d4f2978624-home\") pod \"f0e799dc-85cf-49b1-8486-c1d4f2978624\" (UID: \"f0e799dc-85cf-49b1-8486-c1d4f2978624\") "
Sep 12 00:55:37.157076 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:37.156994    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e799dc-85cf-49b1-8486-c1d4f2978624-model-cache" (OuterVolumeSpecName: "model-cache") pod "f0e799dc-85cf-49b1-8486-c1d4f2978624" (UID: "f0e799dc-85cf-49b1-8486-c1d4f2978624"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:55:37.157209 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:37.157079    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e799dc-85cf-49b1-8486-c1d4f2978624-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f0e799dc-85cf-49b1-8486-c1d4f2978624" (UID: "f0e799dc-85cf-49b1-8486-c1d4f2978624"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:55:37.157267 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:37.157219    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e799dc-85cf-49b1-8486-c1d4f2978624-home" (OuterVolumeSpecName: "home") pod "f0e799dc-85cf-49b1-8486-c1d4f2978624" (UID: "f0e799dc-85cf-49b1-8486-c1d4f2978624"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:55:37.157449 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:37.157373    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0e799dc-85cf-49b1-8486-c1d4f2978624-home\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:55:37.157449 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:37.157397    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0e799dc-85cf-49b1-8486-c1d4f2978624-model-cache\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:55:37.157449 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:37.157414    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0e799dc-85cf-49b1-8486-c1d4f2978624-tmp-dir\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:55:37.159134 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:37.159113    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0e799dc-85cf-49b1-8486-c1d4f2978624-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f0e799dc-85cf-49b1-8486-c1d4f2978624" (UID: "f0e799dc-85cf-49b1-8486-c1d4f2978624"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:55:37.159249 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:37.159146    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e799dc-85cf-49b1-8486-c1d4f2978624-dshm" (OuterVolumeSpecName: "dshm") pod "f0e799dc-85cf-49b1-8486-c1d4f2978624" (UID: "f0e799dc-85cf-49b1-8486-c1d4f2978624"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:55:37.159520 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:37.159488    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0e799dc-85cf-49b1-8486-c1d4f2978624-kube-api-access-x7xg4" (OuterVolumeSpecName: "kube-api-access-x7xg4") pod "f0e799dc-85cf-49b1-8486-c1d4f2978624" (UID: "f0e799dc-85cf-49b1-8486-c1d4f2978624"). InnerVolumeSpecName "kube-api-access-x7xg4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:55:37.178147 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:37.178122    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:37.259010 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:37.258920    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7xg4\" (UniqueName: \"kubernetes.io/projected/f0e799dc-85cf-49b1-8486-c1d4f2978624-kube-api-access-x7xg4\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:55:37.259010 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:37.258950    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0e799dc-85cf-49b1-8486-c1d4f2978624-tls-certs\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:55:37.259010 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:37.258966    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0e799dc-85cf-49b1-8486-c1d4f2978624-dshm\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:55:37.321991 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:37.321955    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s"]
Sep 12 00:55:37.323941 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:37.323922    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-876808/e2e-conflict-b-f937eb5b-kserve-747c6bdb6-w6m4s"]
Sep 12 00:55:37.740959 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:37.740918    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0e799dc-85cf-49b1-8486-c1d4f2978624" path="/var/lib/kubelet/pods/f0e799dc-85cf-49b1-8486-c1d4f2978624/volumes"
Sep 12 00:55:37.892612 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:37.892566    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:38.622162 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:38.622124    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:39.030821 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:39.030780    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf"
Sep 12 00:55:39.031018 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:39.030845    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf"
Sep 12 00:55:39.043439 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:39.043411    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf"
Sep 12 00:55:39.503348 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:39.503316    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:40.040728 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:40.040697    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf"
Sep 12 00:55:40.205965 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:40.205925    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:49.318633 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:49.318599    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc"]
Sep 12 00:55:49.319045 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:49.318849    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc" podUID="6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0" containerName="maas-api" containerID="cri-o://40dd363059d107a27779b9aded7388f0125d54026b2e11ced8dc87aa1a200a4a" gracePeriod=30
Sep 12 00:55:49.347520 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:49.347488    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:49.381024 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:49.380989    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5306e6ca-7986884b4b-f9k62"]
Sep 12 00:55:49.381273 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:49.381244    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-5306e6ca-7986884b4b-f9k62" podUID="0e7c3936-0cce-4fa0-8d12-4c41e89203cd" containerName="payload-pre-processing" containerID="cri-o://ac534fe28f3d86ea179c39824a6d015a19c777b7ce29cab60856bff25744f3e8" gracePeriod=30
Sep 12 00:55:49.452426 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:49.452375    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:49.552798 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:49.550379    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:49.611443 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:49.611416    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:49.641659 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:49.641634    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc"
Sep 12 00:55:49.681273 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:49.681235    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0-openshift-service-ca\") pod \"6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0\" (UID: \"6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0\") "
Sep 12 00:55:49.681412 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:49.681286    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0-metrics-certs\") pod \"6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0\" (UID: \"6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0\") "
Sep 12 00:55:49.681412 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:49.681391    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxwzc\" (UniqueName: \"kubernetes.io/projected/6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0-kube-api-access-mxwzc\") pod \"6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0\" (UID: \"6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0\") "
Sep 12 00:55:49.681546 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:49.681420    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0-maas-api-tls\") pod \"6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0\" (UID: \"6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0\") "
Sep 12 00:55:49.681632 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:49.681606    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0" (UID: "6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:55:49.681846 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:49.681823    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0-openshift-service-ca\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:55:49.684587 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:49.684556    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0-kube-api-access-mxwzc" (OuterVolumeSpecName: "kube-api-access-mxwzc") pod "6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0" (UID: "6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0"). InnerVolumeSpecName "kube-api-access-mxwzc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:55:49.684587 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:49.684555    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0" (UID: "6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:55:49.684766 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:49.684690    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0" (UID: "6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:55:49.686723 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:49.686694    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-5306e6ca-7986884b4b-f9k62"
Sep 12 00:55:49.782381 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:49.782347    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qp2zd\" (UniqueName: \"kubernetes.io/projected/0e7c3936-0cce-4fa0-8d12-4c41e89203cd-kube-api-access-qp2zd\") pod \"0e7c3936-0cce-4fa0-8d12-4c41e89203cd\" (UID: \"0e7c3936-0cce-4fa0-8d12-4c41e89203cd\") "
Sep 12 00:55:49.782557 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:49.782448    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e7c3936-0cce-4fa0-8d12-4c41e89203cd-plugins-config-volume\") pod \"0e7c3936-0cce-4fa0-8d12-4c41e89203cd\" (UID: \"0e7c3936-0cce-4fa0-8d12-4c41e89203cd\") "
Sep 12 00:55:49.782679 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:49.782665    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxwzc\" (UniqueName: \"kubernetes.io/projected/6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0-kube-api-access-mxwzc\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:55:49.782716 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:49.782682    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0-maas-api-tls\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:55:49.782716 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:49.782691    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0-metrics-certs\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:55:49.782817 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:49.782793    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e7c3936-0cce-4fa0-8d12-4c41e89203cd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0e7c3936-0cce-4fa0-8d12-4c41e89203cd" (UID: "0e7c3936-0cce-4fa0-8d12-4c41e89203cd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:55:49.784612 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:49.784591    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e7c3936-0cce-4fa0-8d12-4c41e89203cd-kube-api-access-qp2zd" (OuterVolumeSpecName: "kube-api-access-qp2zd") pod "0e7c3936-0cce-4fa0-8d12-4c41e89203cd" (UID: "0e7c3936-0cce-4fa0-8d12-4c41e89203cd"). InnerVolumeSpecName "kube-api-access-qp2zd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:55:49.883988 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:49.883908    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e7c3936-0cce-4fa0-8d12-4c41e89203cd-plugins-config-volume\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:55:49.883988 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:49.883937    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qp2zd\" (UniqueName: \"kubernetes.io/projected/0e7c3936-0cce-4fa0-8d12-4c41e89203cd-kube-api-access-qp2zd\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:55:49.978665 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:49.978631    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:50.091138 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:50.091103    2558 generic.go:356] "Generic (PLEG): container finished" podID="0e7c3936-0cce-4fa0-8d12-4c41e89203cd" containerID="ac534fe28f3d86ea179c39824a6d015a19c777b7ce29cab60856bff25744f3e8" exitCode=0
Sep 12 00:55:50.091337 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:50.091153    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-5306e6ca-7986884b4b-f9k62"
Sep 12 00:55:50.091337 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:50.091185    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-5306e6ca-7986884b4b-f9k62" event={"ID":"0e7c3936-0cce-4fa0-8d12-4c41e89203cd","Type":"ContainerDied","Data":"ac534fe28f3d86ea179c39824a6d015a19c777b7ce29cab60856bff25744f3e8"}
Sep 12 00:55:50.091337 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:50.091253    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-5306e6ca-7986884b4b-f9k62" event={"ID":"0e7c3936-0cce-4fa0-8d12-4c41e89203cd","Type":"ContainerDied","Data":"09b0934a92bf6ddf54165237a94090a9d29af393fa807b326c709d7baf12cfed"}
Sep 12 00:55:50.091337 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:50.091281    2558 scope.go:117] "RemoveContainer" containerID="ac534fe28f3d86ea179c39824a6d015a19c777b7ce29cab60856bff25744f3e8"
Sep 12 00:55:50.092901 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:50.092878    2558 generic.go:356] "Generic (PLEG): container finished" podID="6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0" containerID="40dd363059d107a27779b9aded7388f0125d54026b2e11ced8dc87aa1a200a4a" exitCode=0
Sep 12 00:55:50.093007 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:50.092944    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc"
Sep 12 00:55:50.093007 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:50.092949    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc" event={"ID":"6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0","Type":"ContainerDied","Data":"40dd363059d107a27779b9aded7388f0125d54026b2e11ced8dc87aa1a200a4a"}
Sep 12 00:55:50.093130 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:50.093014    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc" event={"ID":"6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0","Type":"ContainerDied","Data":"b843f99c690571890097731f16cbce74d4d77e6b2b6bb0787d619e9bd5fe76e1"}
Sep 12 00:55:50.101897 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:50.101876    2558 scope.go:117] "RemoveContainer" containerID="ac534fe28f3d86ea179c39824a6d015a19c777b7ce29cab60856bff25744f3e8"
Sep 12 00:55:50.102227 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:55:50.102207    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac534fe28f3d86ea179c39824a6d015a19c777b7ce29cab60856bff25744f3e8\": container with ID starting with ac534fe28f3d86ea179c39824a6d015a19c777b7ce29cab60856bff25744f3e8 not found: ID does not exist" containerID="ac534fe28f3d86ea179c39824a6d015a19c777b7ce29cab60856bff25744f3e8"
Sep 12 00:55:50.102282 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:50.102236    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac534fe28f3d86ea179c39824a6d015a19c777b7ce29cab60856bff25744f3e8"} err="failed to get container status \"ac534fe28f3d86ea179c39824a6d015a19c777b7ce29cab60856bff25744f3e8\": rpc error: code = NotFound desc = could not find container \"ac534fe28f3d86ea179c39824a6d015a19c777b7ce29cab60856bff25744f3e8\": container with ID starting with ac534fe28f3d86ea179c39824a6d015a19c777b7ce29cab60856bff25744f3e8 not found: ID does not exist"
Sep 12 00:55:50.102282 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:50.102253    2558 scope.go:117] "RemoveContainer" containerID="40dd363059d107a27779b9aded7388f0125d54026b2e11ced8dc87aa1a200a4a"
Sep 12 00:55:50.112931 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:50.112909    2558 scope.go:117] "RemoveContainer" containerID="40dd363059d107a27779b9aded7388f0125d54026b2e11ced8dc87aa1a200a4a"
Sep 12 00:55:50.113312 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:55:50.113291    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40dd363059d107a27779b9aded7388f0125d54026b2e11ced8dc87aa1a200a4a\": container with ID starting with 40dd363059d107a27779b9aded7388f0125d54026b2e11ced8dc87aa1a200a4a not found: ID does not exist" containerID="40dd363059d107a27779b9aded7388f0125d54026b2e11ced8dc87aa1a200a4a"
Sep 12 00:55:50.113385 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:50.113322    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40dd363059d107a27779b9aded7388f0125d54026b2e11ced8dc87aa1a200a4a"} err="failed to get container status \"40dd363059d107a27779b9aded7388f0125d54026b2e11ced8dc87aa1a200a4a\": rpc error: code = NotFound desc = could not find container \"40dd363059d107a27779b9aded7388f0125d54026b2e11ced8dc87aa1a200a4a\": container with ID starting with 40dd363059d107a27779b9aded7388f0125d54026b2e11ced8dc87aa1a200a4a not found: ID does not exist"
Sep 12 00:55:50.114591 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:50.114573    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc"]
Sep 12 00:55:50.119514 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:50.119483    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5306e6ca-5f5f44976d-vlvqc"]
Sep 12 00:55:50.130682 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:50.130645    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5306e6ca-7986884b4b-f9k62"]
Sep 12 00:55:50.132657 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:50.132636    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5306e6ca-7986884b4b-f9k62"]
Sep 12 00:55:50.702577 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:50.702518    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:51.712680 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:51.712650    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:51.739286 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:51.739257    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e7c3936-0cce-4fa0-8d12-4c41e89203cd" path="/var/lib/kubelet/pods/0e7c3936-0cce-4fa0-8d12-4c41e89203cd/volumes"
Sep 12 00:55:51.739688 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:51.739673    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0" path="/var/lib/kubelet/pods/6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0/volumes"
Sep 12 00:55:52.590472 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:52.590432    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:52.689718 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:52.689683    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:53.297346 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:53.297310    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:54.095361 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:54.095320    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:54.525143 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:54.525105    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:55.057808 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:55.057765    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:55:57.633596 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:57.633564    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j"]
Sep 12 00:55:57.729397 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:55:57.729366    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:56:12.653827 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:12.653784    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j" podUID="38bf2c1a-1ee3-48a7-8273-a374436ccc1c" containerName="main" containerID="cri-o://e3cc5b779d1a091be1bf506cc92996b8296800c4ba7898b72c755a5d89eb6a1f" gracePeriod=45
Sep 12 00:56:12.903513 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:12.903487    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j"
Sep 12 00:56:12.917280 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:12.917213    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd"]
Sep 12 00:56:12.917768 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:12.917750    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e7c3936-0cce-4fa0-8d12-4c41e89203cd" containerName="payload-pre-processing"
Sep 12 00:56:12.917875 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:12.917771    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e7c3936-0cce-4fa0-8d12-4c41e89203cd" containerName="payload-pre-processing"
Sep 12 00:56:12.917875 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:12.917781    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0e799dc-85cf-49b1-8486-c1d4f2978624" containerName="main"
Sep 12 00:56:12.917875 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:12.917788    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0e799dc-85cf-49b1-8486-c1d4f2978624" containerName="main"
Sep 12 00:56:12.917875 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:12.917827    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38bf2c1a-1ee3-48a7-8273-a374436ccc1c" containerName="main"
Sep 12 00:56:12.917875 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:12.917835    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="38bf2c1a-1ee3-48a7-8273-a374436ccc1c" containerName="main"
Sep 12 00:56:12.917875 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:12.917848    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0" containerName="maas-api"
Sep 12 00:56:12.917875 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:12.917855    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0" containerName="maas-api"
Sep 12 00:56:12.918237 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:12.917969    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ef36dc5-ee4c-4882-bbb3-e08677dfc7a0" containerName="maas-api"
Sep 12 00:56:12.918237 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:12.917986    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="38bf2c1a-1ee3-48a7-8273-a374436ccc1c" containerName="main"
Sep 12 00:56:12.918237 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:12.917997    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e7c3936-0cce-4fa0-8d12-4c41e89203cd" containerName="payload-pre-processing"
Sep 12 00:56:12.918237 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:12.918010    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0e799dc-85cf-49b1-8486-c1d4f2978624" containerName="main"
Sep 12 00:56:12.921049 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:12.921031    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd"
Sep 12 00:56:12.924283 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:12.924052    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-dd25dd\"/\"auth-test-model-dd25dd-kserve-self-signed-certs\""
Sep 12 00:56:12.924283 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:12.924058    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-dd25dd\"/\"kube-root-ca.crt\""
Sep 12 00:56:12.924283 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:12.924187    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-dd25dd\"/\"openshift-service-ca.crt\""
Sep 12 00:56:12.924523 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:12.924393    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-dd25dd\"/\"default-dockercfg-kjpwh\""
Sep 12 00:56:12.932521 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:12.932496    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd"]
Sep 12 00:56:13.022200 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.022166    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9f622\" (UniqueName: \"kubernetes.io/projected/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-kube-api-access-9f622\") pod \"38bf2c1a-1ee3-48a7-8273-a374436ccc1c\" (UID: \"38bf2c1a-1ee3-48a7-8273-a374436ccc1c\") "
Sep 12 00:56:13.022200 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.022205    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-home\") pod \"38bf2c1a-1ee3-48a7-8273-a374436ccc1c\" (UID: \"38bf2c1a-1ee3-48a7-8273-a374436ccc1c\") "
Sep 12 00:56:13.022428 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.022271    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-tls-certs\") pod \"38bf2c1a-1ee3-48a7-8273-a374436ccc1c\" (UID: \"38bf2c1a-1ee3-48a7-8273-a374436ccc1c\") "
Sep 12 00:56:13.022428 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.022306    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-tmp-dir\") pod \"38bf2c1a-1ee3-48a7-8273-a374436ccc1c\" (UID: \"38bf2c1a-1ee3-48a7-8273-a374436ccc1c\") "
Sep 12 00:56:13.022428 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.022349    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-dshm\") pod \"38bf2c1a-1ee3-48a7-8273-a374436ccc1c\" (UID: \"38bf2c1a-1ee3-48a7-8273-a374436ccc1c\") "
Sep 12 00:56:13.022428 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.022423    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-model-cache\") pod \"38bf2c1a-1ee3-48a7-8273-a374436ccc1c\" (UID: \"38bf2c1a-1ee3-48a7-8273-a374436ccc1c\") "
Sep 12 00:56:13.022646 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.022527    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-home" (OuterVolumeSpecName: "home") pod "38bf2c1a-1ee3-48a7-8273-a374436ccc1c" (UID: "38bf2c1a-1ee3-48a7-8273-a374436ccc1c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:56:13.022646 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.022631    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/910b9e2d-fc1c-49d5-a191-064f56754798-home\") pod \"auth-test-model-dd25dd-kserve-7659474bff-mcxbd\" (UID: \"910b9e2d-fc1c-49d5-a191-064f56754798\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd"
Sep 12 00:56:13.022758 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.022655    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "38bf2c1a-1ee3-48a7-8273-a374436ccc1c" (UID: "38bf2c1a-1ee3-48a7-8273-a374436ccc1c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:56:13.022758 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.022683    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/910b9e2d-fc1c-49d5-a191-064f56754798-dshm\") pod \"auth-test-model-dd25dd-kserve-7659474bff-mcxbd\" (UID: \"910b9e2d-fc1c-49d5-a191-064f56754798\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd"
Sep 12 00:56:13.022758 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.022714    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/910b9e2d-fc1c-49d5-a191-064f56754798-model-cache\") pod \"auth-test-model-dd25dd-kserve-7659474bff-mcxbd\" (UID: \"910b9e2d-fc1c-49d5-a191-064f56754798\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd"
Sep 12 00:56:13.022758 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.022708    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-model-cache" (OuterVolumeSpecName: "model-cache") pod "38bf2c1a-1ee3-48a7-8273-a374436ccc1c" (UID: "38bf2c1a-1ee3-48a7-8273-a374436ccc1c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:56:13.022965 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.022781    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/910b9e2d-fc1c-49d5-a191-064f56754798-tmp-dir\") pod \"auth-test-model-dd25dd-kserve-7659474bff-mcxbd\" (UID: \"910b9e2d-fc1c-49d5-a191-064f56754798\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd"
Sep 12 00:56:13.022965 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.022824    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52z44\" (UniqueName: \"kubernetes.io/projected/910b9e2d-fc1c-49d5-a191-064f56754798-kube-api-access-52z44\") pod \"auth-test-model-dd25dd-kserve-7659474bff-mcxbd\" (UID: \"910b9e2d-fc1c-49d5-a191-064f56754798\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd"
Sep 12 00:56:13.022965 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.022874    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/910b9e2d-fc1c-49d5-a191-064f56754798-tls-certs\") pod \"auth-test-model-dd25dd-kserve-7659474bff-mcxbd\" (UID: \"910b9e2d-fc1c-49d5-a191-064f56754798\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd"
Sep 12 00:56:13.023152 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.023069    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-model-cache\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:56:13.023152 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.023121    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-home\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:56:13.023152 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.023136    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-tmp-dir\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:56:13.024725 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.024703    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-kube-api-access-9f622" (OuterVolumeSpecName: "kube-api-access-9f622") pod "38bf2c1a-1ee3-48a7-8273-a374436ccc1c" (UID: "38bf2c1a-1ee3-48a7-8273-a374436ccc1c"). InnerVolumeSpecName "kube-api-access-9f622". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:56:13.025121 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.025099    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-dshm" (OuterVolumeSpecName: "dshm") pod "38bf2c1a-1ee3-48a7-8273-a374436ccc1c" (UID: "38bf2c1a-1ee3-48a7-8273-a374436ccc1c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:56:13.025203 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.025128    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "38bf2c1a-1ee3-48a7-8273-a374436ccc1c" (UID: "38bf2c1a-1ee3-48a7-8273-a374436ccc1c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:56:13.123251 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.123213    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:56:13.123697 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.123674    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/910b9e2d-fc1c-49d5-a191-064f56754798-home\") pod \"auth-test-model-dd25dd-kserve-7659474bff-mcxbd\" (UID: \"910b9e2d-fc1c-49d5-a191-064f56754798\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd"
Sep 12 00:56:13.123768 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.123710    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/910b9e2d-fc1c-49d5-a191-064f56754798-dshm\") pod \"auth-test-model-dd25dd-kserve-7659474bff-mcxbd\" (UID: \"910b9e2d-fc1c-49d5-a191-064f56754798\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd"
Sep 12 00:56:13.123768 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.123731    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/910b9e2d-fc1c-49d5-a191-064f56754798-model-cache\") pod \"auth-test-model-dd25dd-kserve-7659474bff-mcxbd\" (UID: \"910b9e2d-fc1c-49d5-a191-064f56754798\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd"
Sep 12 00:56:13.123768 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.123749    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/910b9e2d-fc1c-49d5-a191-064f56754798-tmp-dir\") pod \"auth-test-model-dd25dd-kserve-7659474bff-mcxbd\" (UID: \"910b9e2d-fc1c-49d5-a191-064f56754798\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd"
Sep 12 00:56:13.123924 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.123779    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52z44\" (UniqueName: \"kubernetes.io/projected/910b9e2d-fc1c-49d5-a191-064f56754798-kube-api-access-52z44\") pod \"auth-test-model-dd25dd-kserve-7659474bff-mcxbd\" (UID: \"910b9e2d-fc1c-49d5-a191-064f56754798\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd"
Sep 12 00:56:13.123924 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.123804    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/910b9e2d-fc1c-49d5-a191-064f56754798-tls-certs\") pod \"auth-test-model-dd25dd-kserve-7659474bff-mcxbd\" (UID: \"910b9e2d-fc1c-49d5-a191-064f56754798\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd"
Sep 12 00:56:13.123924 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.123911    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-dshm\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:56:13.124064 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.123923    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9f622\" (UniqueName: \"kubernetes.io/projected/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-kube-api-access-9f622\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:56:13.124064 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.123937    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38bf2c1a-1ee3-48a7-8273-a374436ccc1c-tls-certs\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:56:13.124202 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.124178    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/910b9e2d-fc1c-49d5-a191-064f56754798-model-cache\") pod \"auth-test-model-dd25dd-kserve-7659474bff-mcxbd\" (UID: \"910b9e2d-fc1c-49d5-a191-064f56754798\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd"
Sep 12 00:56:13.124253 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.124205    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/910b9e2d-fc1c-49d5-a191-064f56754798-home\") pod \"auth-test-model-dd25dd-kserve-7659474bff-mcxbd\" (UID: \"910b9e2d-fc1c-49d5-a191-064f56754798\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd"
Sep 12 00:56:13.124303 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.124262    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/910b9e2d-fc1c-49d5-a191-064f56754798-tmp-dir\") pod \"auth-test-model-dd25dd-kserve-7659474bff-mcxbd\" (UID: \"910b9e2d-fc1c-49d5-a191-064f56754798\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd"
Sep 12 00:56:13.126184 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.126163    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/910b9e2d-fc1c-49d5-a191-064f56754798-dshm\") pod \"auth-test-model-dd25dd-kserve-7659474bff-mcxbd\" (UID: \"910b9e2d-fc1c-49d5-a191-064f56754798\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd"
Sep 12 00:56:13.126443 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.126424    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/910b9e2d-fc1c-49d5-a191-064f56754798-tls-certs\") pod \"auth-test-model-dd25dd-kserve-7659474bff-mcxbd\" (UID: \"910b9e2d-fc1c-49d5-a191-064f56754798\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd"
Sep 12 00:56:13.132159 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.132142    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52z44\" (UniqueName: \"kubernetes.io/projected/910b9e2d-fc1c-49d5-a191-064f56754798-kube-api-access-52z44\") pod \"auth-test-model-dd25dd-kserve-7659474bff-mcxbd\" (UID: \"910b9e2d-fc1c-49d5-a191-064f56754798\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd"
Sep 12 00:56:13.216450 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.216409    2558 generic.go:356] "Generic (PLEG): container finished" podID="38bf2c1a-1ee3-48a7-8273-a374436ccc1c" containerID="e3cc5b779d1a091be1bf506cc92996b8296800c4ba7898b72c755a5d89eb6a1f" exitCode=0
Sep 12 00:56:13.216657 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.216475    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j"
Sep 12 00:56:13.216657 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.216501    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j" event={"ID":"38bf2c1a-1ee3-48a7-8273-a374436ccc1c","Type":"ContainerDied","Data":"e3cc5b779d1a091be1bf506cc92996b8296800c4ba7898b72c755a5d89eb6a1f"}
Sep 12 00:56:13.216657 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.216560    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j" event={"ID":"38bf2c1a-1ee3-48a7-8273-a374436ccc1c","Type":"ContainerDied","Data":"22b519fce630e2bc36ab606d8f2c98d4f3638059f59518fbce2751d75bf56a0e"}
Sep 12 00:56:13.216657 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.216594    2558 scope.go:117] "RemoveContainer" containerID="e3cc5b779d1a091be1bf506cc92996b8296800c4ba7898b72c755a5d89eb6a1f"
Sep 12 00:56:13.226891 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.226872    2558 scope.go:117] "RemoveContainer" containerID="e3cc5b779d1a091be1bf506cc92996b8296800c4ba7898b72c755a5d89eb6a1f"
Sep 12 00:56:13.227315 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:56:13.227294    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3cc5b779d1a091be1bf506cc92996b8296800c4ba7898b72c755a5d89eb6a1f\": container with ID starting with e3cc5b779d1a091be1bf506cc92996b8296800c4ba7898b72c755a5d89eb6a1f not found: ID does not exist" containerID="e3cc5b779d1a091be1bf506cc92996b8296800c4ba7898b72c755a5d89eb6a1f"
Sep 12 00:56:13.227391 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.227324    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3cc5b779d1a091be1bf506cc92996b8296800c4ba7898b72c755a5d89eb6a1f"} err="failed to get container status \"e3cc5b779d1a091be1bf506cc92996b8296800c4ba7898b72c755a5d89eb6a1f\": rpc error: code = NotFound desc = could not find container \"e3cc5b779d1a091be1bf506cc92996b8296800c4ba7898b72c755a5d89eb6a1f\": container with ID starting with e3cc5b779d1a091be1bf506cc92996b8296800c4ba7898b72c755a5d89eb6a1f not found: ID does not exist"
Sep 12 00:56:13.233113 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.233073    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd"
Sep 12 00:56:13.240707 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.240686    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j"]
Sep 12 00:56:13.246266 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.246246    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-5306e6ca/e2e-lifecycle-model-5306e6ca-kserve-6b8fddb547-6hv7j"]
Sep 12 00:56:13.507841 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.507807    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd"]
Sep 12 00:56:13.510220 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:56:13.510194    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod910b9e2d_fc1c_49d5_a191_064f56754798.slice/crio-163b9d0e1427552c62cf12f9b0d5896260ee92856809f50ef22145a643e12b0d WatchSource:0}: Error finding container 163b9d0e1427552c62cf12f9b0d5896260ee92856809f50ef22145a643e12b0d: Status 404 returned error can't find the container with id 163b9d0e1427552c62cf12f9b0d5896260ee92856809f50ef22145a643e12b0d
Sep 12 00:56:13.738938 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:13.738897    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38bf2c1a-1ee3-48a7-8273-a374436ccc1c" path="/var/lib/kubelet/pods/38bf2c1a-1ee3-48a7-8273-a374436ccc1c/volumes"
Sep 12 00:56:14.224776 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:14.224736    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd" event={"ID":"910b9e2d-fc1c-49d5-a191-064f56754798","Type":"ContainerStarted","Data":"410b0911dbfd89469bac94e9bade963366112a583dae567b34d41c19ac96a393"}
Sep 12 00:56:14.224964 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:14.224784    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd" event={"ID":"910b9e2d-fc1c-49d5-a191-064f56754798","Type":"ContainerStarted","Data":"163b9d0e1427552c62cf12f9b0d5896260ee92856809f50ef22145a643e12b0d"}
Sep 12 00:56:14.243157 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:14.243074    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd" podStartSLOduration=2.243052435 podStartE2EDuration="2.243052435s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:56:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:56:14.241899119 +0000 UTC m=+1610.915957978" watchObservedRunningTime="2026-09-12 00:56:14.243052435 +0000 UTC m=+1610.917111297"
Sep 12 00:56:21.193102 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:21.193011    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:56:21.261311 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:21.261278    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:56:23.233909 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:23.233876    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd"
Sep 12 00:56:23.234439 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:23.233932    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd"
Sep 12 00:56:23.246688 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:23.246663    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd"
Sep 12 00:56:23.283556 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:23.283529    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd"
Sep 12 00:56:34.614311 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:34.614275    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:56:34.921735 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:34.921603    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:56:38.839424 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:38.839386    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:56:40.915429 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:40.915379    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:56:41.179520 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:41.179394    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:56:46.052906 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:46.052871    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:56:46.713211 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:46.713178    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:56:46.821362 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:46.821311    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:56:47.052296 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:47.052203    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:56:48.026391 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:48.026354    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk"]
Sep 12 00:56:48.030247 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:48.030228    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk"
Sep 12 00:56:48.034279 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:48.034256    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-876808\"/\"openshift-service-ca.crt\""
Sep 12 00:56:48.034423 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:48.034298    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-876808\"/\"kube-root-ca.crt\""
Sep 12 00:56:48.034423 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:48.034306    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-876808\"/\"distinct-2-w4-876808-kserve-self-signed-certs\""
Sep 12 00:56:48.034423 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:48.034301    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-876808\"/\"default-dockercfg-2c5sq\""
Sep 12 00:56:48.045848 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:48.045826    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk"]
Sep 12 00:56:48.148930 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:48.148895    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:56:48.190938 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:48.190903    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bfq2\" (UniqueName: \"kubernetes.io/projected/4c294f5a-088e-4633-829d-ed229ff054e9-kube-api-access-2bfq2\") pod \"distinct-2-w4-876808-kserve-5c45757dff-d8jmk\" (UID: \"4c294f5a-088e-4633-829d-ed229ff054e9\") " pod="e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk"
Sep 12 00:56:48.191196 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:48.190946    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c294f5a-088e-4633-829d-ed229ff054e9-home\") pod \"distinct-2-w4-876808-kserve-5c45757dff-d8jmk\" (UID: \"4c294f5a-088e-4633-829d-ed229ff054e9\") " pod="e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk"
Sep 12 00:56:48.191196 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:48.191125    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c294f5a-088e-4633-829d-ed229ff054e9-model-cache\") pod \"distinct-2-w4-876808-kserve-5c45757dff-d8jmk\" (UID: \"4c294f5a-088e-4633-829d-ed229ff054e9\") " pod="e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk"
Sep 12 00:56:48.191196 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:48.191178    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c294f5a-088e-4633-829d-ed229ff054e9-tmp-dir\") pod \"distinct-2-w4-876808-kserve-5c45757dff-d8jmk\" (UID: \"4c294f5a-088e-4633-829d-ed229ff054e9\") " pod="e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk"
Sep 12 00:56:48.191377 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:48.191219    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c294f5a-088e-4633-829d-ed229ff054e9-tls-certs\") pod \"distinct-2-w4-876808-kserve-5c45757dff-d8jmk\" (UID: \"4c294f5a-088e-4633-829d-ed229ff054e9\") " pod="e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk"
Sep 12 00:56:48.191377 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:48.191306    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c294f5a-088e-4633-829d-ed229ff054e9-dshm\") pod \"distinct-2-w4-876808-kserve-5c45757dff-d8jmk\" (UID: \"4c294f5a-088e-4633-829d-ed229ff054e9\") " pod="e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk"
Sep 12 00:56:48.292174 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:48.292047    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c294f5a-088e-4633-829d-ed229ff054e9-model-cache\") pod \"distinct-2-w4-876808-kserve-5c45757dff-d8jmk\" (UID: \"4c294f5a-088e-4633-829d-ed229ff054e9\") " pod="e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk"
Sep 12 00:56:48.292174 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:48.292101    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c294f5a-088e-4633-829d-ed229ff054e9-tmp-dir\") pod \"distinct-2-w4-876808-kserve-5c45757dff-d8jmk\" (UID: \"4c294f5a-088e-4633-829d-ed229ff054e9\") " pod="e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk"
Sep 12 00:56:48.292418 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:48.292223    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c294f5a-088e-4633-829d-ed229ff054e9-tls-certs\") pod \"distinct-2-w4-876808-kserve-5c45757dff-d8jmk\" (UID: \"4c294f5a-088e-4633-829d-ed229ff054e9\") " pod="e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk"
Sep 12 00:56:48.292418 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:48.292326    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c294f5a-088e-4633-829d-ed229ff054e9-dshm\") pod \"distinct-2-w4-876808-kserve-5c45757dff-d8jmk\" (UID: \"4c294f5a-088e-4633-829d-ed229ff054e9\") " pod="e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk"
Sep 12 00:56:48.292418 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:48.292403    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bfq2\" (UniqueName: \"kubernetes.io/projected/4c294f5a-088e-4633-829d-ed229ff054e9-kube-api-access-2bfq2\") pod \"distinct-2-w4-876808-kserve-5c45757dff-d8jmk\" (UID: \"4c294f5a-088e-4633-829d-ed229ff054e9\") " pod="e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk"
Sep 12 00:56:48.292575 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:48.292437    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c294f5a-088e-4633-829d-ed229ff054e9-home\") pod \"distinct-2-w4-876808-kserve-5c45757dff-d8jmk\" (UID: \"4c294f5a-088e-4633-829d-ed229ff054e9\") " pod="e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk"
Sep 12 00:56:48.292575 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:48.292447    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c294f5a-088e-4633-829d-ed229ff054e9-tmp-dir\") pod \"distinct-2-w4-876808-kserve-5c45757dff-d8jmk\" (UID: \"4c294f5a-088e-4633-829d-ed229ff054e9\") " pod="e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk"
Sep 12 00:56:48.292575 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:48.292523    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c294f5a-088e-4633-829d-ed229ff054e9-model-cache\") pod \"distinct-2-w4-876808-kserve-5c45757dff-d8jmk\" (UID: \"4c294f5a-088e-4633-829d-ed229ff054e9\") " pod="e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk"
Sep 12 00:56:48.292715 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:48.292691    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c294f5a-088e-4633-829d-ed229ff054e9-home\") pod \"distinct-2-w4-876808-kserve-5c45757dff-d8jmk\" (UID: \"4c294f5a-088e-4633-829d-ed229ff054e9\") " pod="e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk"
Sep 12 00:56:48.294685 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:48.294660    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c294f5a-088e-4633-829d-ed229ff054e9-dshm\") pod \"distinct-2-w4-876808-kserve-5c45757dff-d8jmk\" (UID: \"4c294f5a-088e-4633-829d-ed229ff054e9\") " pod="e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk"
Sep 12 00:56:48.295013 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:48.294994    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c294f5a-088e-4633-829d-ed229ff054e9-tls-certs\") pod \"distinct-2-w4-876808-kserve-5c45757dff-d8jmk\" (UID: \"4c294f5a-088e-4633-829d-ed229ff054e9\") " pod="e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk"
Sep 12 00:56:48.304835 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:48.304809    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bfq2\" (UniqueName: \"kubernetes.io/projected/4c294f5a-088e-4633-829d-ed229ff054e9-kube-api-access-2bfq2\") pod \"distinct-2-w4-876808-kserve-5c45757dff-d8jmk\" (UID: \"4c294f5a-088e-4633-829d-ed229ff054e9\") " pod="e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk"
Sep 12 00:56:48.339420 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:48.339375    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk"
Sep 12 00:56:48.456010 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:48.455966    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:56:48.594600 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:48.594513    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:56:48.628579 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:48.628399    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk"]
Sep 12 00:56:48.631397 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:56:48.631370    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c294f5a_088e_4633_829d_ed229ff054e9.slice/crio-fc8288effc49c4c8c10f0edd07c375fd48bbdbc839a327f348a7745a499f01ff WatchSource:0}: Error finding container fc8288effc49c4c8c10f0edd07c375fd48bbdbc839a327f348a7745a499f01ff: Status 404 returned error can't find the container with id fc8288effc49c4c8c10f0edd07c375fd48bbdbc839a327f348a7745a499f01ff
Sep 12 00:56:49.420100 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:49.420048    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk" event={"ID":"4c294f5a-088e-4633-829d-ed229ff054e9","Type":"ContainerStarted","Data":"93a7bebba33b96860a5c0270304aa6d6e9ff94d11fd58b9f0d53d864f7c6e68e"}
Sep 12 00:56:49.420497 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:49.420107    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk" event={"ID":"4c294f5a-088e-4633-829d-ed229ff054e9","Type":"ContainerStarted","Data":"fc8288effc49c4c8c10f0edd07c375fd48bbdbc839a327f348a7745a499f01ff"}
Sep 12 00:56:49.424796 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:49.424768    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:56:49.466147 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:49.466071    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk" podStartSLOduration=1.466053107 podStartE2EDuration="1.466053107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:56:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:56:49.464034037 +0000 UTC m=+1646.138092897" watchObservedRunningTime="2026-09-12 00:56:49.466053107 +0000 UTC m=+1646.140111999"
Sep 12 00:56:49.548620 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:49.548587    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:56:50.908553 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:50.908514    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:56:57.126508 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:57.126465    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:56:57.225826 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:57.225788    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:56:57.784829 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:57.784787    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:56:58.340133 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:58.340076    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk"
Sep 12 00:56:58.340534 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:58.340152    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk"
Sep 12 00:56:58.352727 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:58.352705    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk"
Sep 12 00:56:58.478565 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:56:58.478540    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk"
Sep 12 00:57:04.381841 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:04.381809    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:57:04.533206 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:04.533172    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:57:04.701687 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:04.701654    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:57:07.967319 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:07.967283    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:57:15.113545 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:15.113513    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:57:15.348498 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:15.348462    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:57:36.249591 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:36.249557    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:57:36.827789 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:36.827761    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:57:42.555584 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:42.555535    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:57:42.632455 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:42.632387    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:57:42.893643 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:42.893544    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:57:43.190040 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:43.190007    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:57:43.473602 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:43.473518    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:57:45.371483 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:45.371403    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:57:45.982039 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:45.981993    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:57:46.074559 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:46.074528    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:57:46.299657 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:46.299545    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:57:46.439572 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:46.439530    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:57:47.289670 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:47.289631    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:57:47.454021 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:47.453980    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:57:47.563153 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:47.563019    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:57:48.242909 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:48.242863    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:57:49.155730 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:49.155688    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:57:49.986566 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:49.986523    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:57:50.784764 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:50.784731    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:57:51.632745 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:51.632706    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:57:51.722393 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:51.722364    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:57:51.849632 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:51.849602    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:57:52.184198 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:52.184164    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:57:52.255804 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:52.255763    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:57:52.322824 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:52.322792    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-04f8e8-6fc7764877-9b8c6"]
Sep 12 00:57:52.326862 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:52.326844    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-04f8e8-6fc7764877-9b8c6"
Sep 12 00:57:52.329423 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:52.329402    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-04f8e8-dockercfg-mg845\""
Sep 12 00:57:52.330445 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:52.330427    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-04f8e8\""
Sep 12 00:57:52.338545 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:52.338519    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-04f8e8-6fc7764877-9b8c6"]
Sep 12 00:57:52.464760 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:52.464719    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14c9cff3-3757-4daf-837e-cbb86a3fa5b7-plugins-config-volume\") pod \"payload-processing-e2e-api-a-04f8e8-6fc7764877-9b8c6\" (UID: \"14c9cff3-3757-4daf-837e-cbb86a3fa5b7\") " pod="openshift-ingress/payload-processing-e2e-api-a-04f8e8-6fc7764877-9b8c6"
Sep 12 00:57:52.464950 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:52.464788    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkq55\" (UniqueName: \"kubernetes.io/projected/14c9cff3-3757-4daf-837e-cbb86a3fa5b7-kube-api-access-zkq55\") pod \"payload-processing-e2e-api-a-04f8e8-6fc7764877-9b8c6\" (UID: \"14c9cff3-3757-4daf-837e-cbb86a3fa5b7\") " pod="openshift-ingress/payload-processing-e2e-api-a-04f8e8-6fc7764877-9b8c6"
Sep 12 00:57:52.565468 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:52.565426    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14c9cff3-3757-4daf-837e-cbb86a3fa5b7-plugins-config-volume\") pod \"payload-processing-e2e-api-a-04f8e8-6fc7764877-9b8c6\" (UID: \"14c9cff3-3757-4daf-837e-cbb86a3fa5b7\") " pod="openshift-ingress/payload-processing-e2e-api-a-04f8e8-6fc7764877-9b8c6"
Sep 12 00:57:52.565651 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:52.565497    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkq55\" (UniqueName: \"kubernetes.io/projected/14c9cff3-3757-4daf-837e-cbb86a3fa5b7-kube-api-access-zkq55\") pod \"payload-processing-e2e-api-a-04f8e8-6fc7764877-9b8c6\" (UID: \"14c9cff3-3757-4daf-837e-cbb86a3fa5b7\") " pod="openshift-ingress/payload-processing-e2e-api-a-04f8e8-6fc7764877-9b8c6"
Sep 12 00:57:52.566197 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:52.566178    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14c9cff3-3757-4daf-837e-cbb86a3fa5b7-plugins-config-volume\") pod \"payload-processing-e2e-api-a-04f8e8-6fc7764877-9b8c6\" (UID: \"14c9cff3-3757-4daf-837e-cbb86a3fa5b7\") " pod="openshift-ingress/payload-processing-e2e-api-a-04f8e8-6fc7764877-9b8c6"
Sep 12 00:57:52.574529 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:52.574506    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkq55\" (UniqueName: \"kubernetes.io/projected/14c9cff3-3757-4daf-837e-cbb86a3fa5b7-kube-api-access-zkq55\") pod \"payload-processing-e2e-api-a-04f8e8-6fc7764877-9b8c6\" (UID: \"14c9cff3-3757-4daf-837e-cbb86a3fa5b7\") " pod="openshift-ingress/payload-processing-e2e-api-a-04f8e8-6fc7764877-9b8c6"
Sep 12 00:57:52.638198 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:52.638162    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-04f8e8-6fc7764877-9b8c6"
Sep 12 00:57:52.918814 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:52.918787    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-04f8e8-6fc7764877-9b8c6"]
Sep 12 00:57:52.921220 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:57:52.921188    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14c9cff3_3757_4daf_837e_cbb86a3fa5b7.slice/crio-76204a370dbe366194958ddeb6a80e34cec0f66a2e99efc9e228987e3e706980 WatchSource:0}: Error finding container 76204a370dbe366194958ddeb6a80e34cec0f66a2e99efc9e228987e3e706980: Status 404 returned error can't find the container with id 76204a370dbe366194958ddeb6a80e34cec0f66a2e99efc9e228987e3e706980
Sep 12 00:57:53.772180 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:53.772143    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-04f8e8-6fc7764877-9b8c6" event={"ID":"14c9cff3-3757-4daf-837e-cbb86a3fa5b7","Type":"ContainerStarted","Data":"aba16ac7a494a1fb8e6e2101396682aaa05802a844611d01823e822fe9d1dc05"}
Sep 12 00:57:53.772379 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:53.772197    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-04f8e8-6fc7764877-9b8c6" event={"ID":"14c9cff3-3757-4daf-837e-cbb86a3fa5b7","Type":"ContainerStarted","Data":"76204a370dbe366194958ddeb6a80e34cec0f66a2e99efc9e228987e3e706980"}
Sep 12 00:57:53.772379 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:53.772272    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-04f8e8-6fc7764877-9b8c6"
Sep 12 00:57:53.792970 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:57:53.792913    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-04f8e8-6fc7764877-9b8c6" podStartSLOduration=1.792894167 podStartE2EDuration="1.792894167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:57:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:57:53.790492835 +0000 UTC m=+1710.464551695" watchObservedRunningTime="2026-09-12 00:57:53.792894167 +0000 UTC m=+1710.466953071"
Sep 12 00:58:04.780872 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:04.780841    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-04f8e8-6fc7764877-9b8c6"
Sep 12 00:58:13.914340 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:13.914297    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:14.945278 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:14.945243    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:15.751405 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:15.751369    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:16.027834 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:16.027742    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:16.462303 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:16.462275    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:16.564251 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:16.564222    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:16.772753 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:16.772665    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:16.879176 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:16.879144    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:17.486752 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:17.486714    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:18.304547 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:18.304514    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:19.194826 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:19.194790    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:20.194323 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:20.194255    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:20.336063 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:20.336033    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:20.438206 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:20.438167    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:21.235572 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:21.235528    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:21.893691 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:21.893662    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c14a20-f885bd68-8krn2"]
Sep 12 00:58:21.898308 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:21.898280    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c14a20-f885bd68-8krn2"
Sep 12 00:58:21.901949 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:21.901928    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-c14a20-serving-cert\""
Sep 12 00:58:21.913124 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:21.913069    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c14a20-f885bd68-8krn2"]
Sep 12 00:58:21.932866 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:21.932830    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:21.968740 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:21.968702    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42381ab7-985d-4f04-ba78-ce3325c6bed8-openshift-service-ca\") pod \"maas-api-e2e-api-b-c14a20-f885bd68-8krn2\" (UID: \"42381ab7-985d-4f04-ba78-ce3325c6bed8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c14a20-f885bd68-8krn2"
Sep 12 00:58:21.968951 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:21.968876    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42381ab7-985d-4f04-ba78-ce3325c6bed8-maas-api-tls\") pod \"maas-api-e2e-api-b-c14a20-f885bd68-8krn2\" (UID: \"42381ab7-985d-4f04-ba78-ce3325c6bed8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c14a20-f885bd68-8krn2"
Sep 12 00:58:21.969018 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:21.968949    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42381ab7-985d-4f04-ba78-ce3325c6bed8-metrics-certs\") pod \"maas-api-e2e-api-b-c14a20-f885bd68-8krn2\" (UID: \"42381ab7-985d-4f04-ba78-ce3325c6bed8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c14a20-f885bd68-8krn2"
Sep 12 00:58:21.969018 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:21.968980    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98zxg\" (UniqueName: \"kubernetes.io/projected/42381ab7-985d-4f04-ba78-ce3325c6bed8-kube-api-access-98zxg\") pod \"maas-api-e2e-api-b-c14a20-f885bd68-8krn2\" (UID: \"42381ab7-985d-4f04-ba78-ce3325c6bed8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c14a20-f885bd68-8krn2"
Sep 12 00:58:22.028521 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:22.028488    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:22.070042 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:22.070000    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42381ab7-985d-4f04-ba78-ce3325c6bed8-maas-api-tls\") pod \"maas-api-e2e-api-b-c14a20-f885bd68-8krn2\" (UID: \"42381ab7-985d-4f04-ba78-ce3325c6bed8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c14a20-f885bd68-8krn2"
Sep 12 00:58:22.070260 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:22.070079    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42381ab7-985d-4f04-ba78-ce3325c6bed8-metrics-certs\") pod \"maas-api-e2e-api-b-c14a20-f885bd68-8krn2\" (UID: \"42381ab7-985d-4f04-ba78-ce3325c6bed8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c14a20-f885bd68-8krn2"
Sep 12 00:58:22.070260 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:22.070136    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98zxg\" (UniqueName: \"kubernetes.io/projected/42381ab7-985d-4f04-ba78-ce3325c6bed8-kube-api-access-98zxg\") pod \"maas-api-e2e-api-b-c14a20-f885bd68-8krn2\" (UID: \"42381ab7-985d-4f04-ba78-ce3325c6bed8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c14a20-f885bd68-8krn2"
Sep 12 00:58:22.070260 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:22.070244    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42381ab7-985d-4f04-ba78-ce3325c6bed8-openshift-service-ca\") pod \"maas-api-e2e-api-b-c14a20-f885bd68-8krn2\" (UID: \"42381ab7-985d-4f04-ba78-ce3325c6bed8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c14a20-f885bd68-8krn2"
Sep 12 00:58:22.070860 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:22.070836    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42381ab7-985d-4f04-ba78-ce3325c6bed8-openshift-service-ca\") pod \"maas-api-e2e-api-b-c14a20-f885bd68-8krn2\" (UID: \"42381ab7-985d-4f04-ba78-ce3325c6bed8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c14a20-f885bd68-8krn2"
Sep 12 00:58:22.072685 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:22.072658    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42381ab7-985d-4f04-ba78-ce3325c6bed8-metrics-certs\") pod \"maas-api-e2e-api-b-c14a20-f885bd68-8krn2\" (UID: \"42381ab7-985d-4f04-ba78-ce3325c6bed8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c14a20-f885bd68-8krn2"
Sep 12 00:58:22.072776 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:22.072754    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42381ab7-985d-4f04-ba78-ce3325c6bed8-maas-api-tls\") pod \"maas-api-e2e-api-b-c14a20-f885bd68-8krn2\" (UID: \"42381ab7-985d-4f04-ba78-ce3325c6bed8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c14a20-f885bd68-8krn2"
Sep 12 00:58:22.078840 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:22.078820    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98zxg\" (UniqueName: \"kubernetes.io/projected/42381ab7-985d-4f04-ba78-ce3325c6bed8-kube-api-access-98zxg\") pod \"maas-api-e2e-api-b-c14a20-f885bd68-8krn2\" (UID: \"42381ab7-985d-4f04-ba78-ce3325c6bed8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c14a20-f885bd68-8krn2"
Sep 12 00:58:22.130431 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:22.130395    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:22.209708 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:22.209674    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c14a20-f885bd68-8krn2"
Sep 12 00:58:22.437378 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:22.437336    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:22.489098 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:22.489053    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c14a20-f885bd68-8krn2"]
Sep 12 00:58:22.490882 ip-10-0-142-85 kubenswrapper[2558]: W0912 00:58:22.490854    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42381ab7_985d_4f04_ba78_ce3325c6bed8.slice/crio-9e12cc1fe33d833f3f25b76249839cfda19b946076d98a25ba3a2dff46449cf6 WatchSource:0}: Error finding container 9e12cc1fe33d833f3f25b76249839cfda19b946076d98a25ba3a2dff46449cf6: Status 404 returned error can't find the container with id 9e12cc1fe33d833f3f25b76249839cfda19b946076d98a25ba3a2dff46449cf6
Sep 12 00:58:22.492296 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:22.492279    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 12 00:58:22.541499 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:22.541472    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:22.876889 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:22.876803    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:22.935182 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:22.935157    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c14a20-f885bd68-8krn2" event={"ID":"42381ab7-985d-4f04-ba78-ce3325c6bed8","Type":"ContainerStarted","Data":"9e12cc1fe33d833f3f25b76249839cfda19b946076d98a25ba3a2dff46449cf6"}
Sep 12 00:58:23.772698 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:23.772648    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:23.942570 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:23.942533    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c14a20-f885bd68-8krn2" event={"ID":"42381ab7-985d-4f04-ba78-ce3325c6bed8","Type":"ContainerStarted","Data":"5f5682d1a6e47d549114239a606d3b12e9f66487075d18c012fff61998a331c0"}
Sep 12 00:58:23.943040 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:23.942711    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c14a20-f885bd68-8krn2"
Sep 12 00:58:23.960186 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:23.960139    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c14a20-f885bd68-8krn2" podStartSLOduration=2.526928792 podStartE2EDuration="2.960121535s" totalImagesPullingTime="433.192743ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 00:58:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 00:58:23.959940094 +0000 UTC m=+1740.633998953" watchObservedRunningTime="2026-09-12 00:58:23.960121535 +0000 UTC m=+1740.634180393"
Sep 12 00:58:24.463322 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:24.463280    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:25.185216 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:25.185182    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:26.016331 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:26.016289    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:26.692594 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:26.692560    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:27.575579 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:27.575533    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:29.954679 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:29.954651    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c14a20-f885bd68-8krn2"
Sep 12 00:58:44.035878 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:44.035835    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:45.078221 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:45.078175    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:45.184249 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:45.184215    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:46.077429 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:46.077394    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:46.886433 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:46.886396    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:47.572718 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:47.572682    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:48.487226 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:48.487194    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:49.284995 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:49.284950    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:49.987322 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:49.987291    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:50.670350 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:50.670296    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:51.498464 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:51.498431    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:52.189652 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:52.189610    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:52.971198 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:52.971105    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:53.646497 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:53.646466    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:54.374174 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:54.374132    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:58:58.097790 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:58:58.097748    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:03.672206 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:03.672173    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:03.703277 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:03.703248    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-04f8e8-6fc7764877-9b8c6"]
Sep 12 00:59:03.703524 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:03.703497    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-04f8e8-6fc7764877-9b8c6" podUID="14c9cff3-3757-4daf-837e-cbb86a3fa5b7" containerName="payload-processing" containerID="cri-o://aba16ac7a494a1fb8e6e2101396682aaa05802a844611d01823e822fe9d1dc05" gracePeriod=30
Sep 12 00:59:03.772289 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:03.772239    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:03.877120 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:03.877068    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:03.961341 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:03.961313    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-04f8e8-6fc7764877-9b8c6"
Sep 12 00:59:04.023931 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:04.023903    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14c9cff3-3757-4daf-837e-cbb86a3fa5b7-plugins-config-volume\") pod \"14c9cff3-3757-4daf-837e-cbb86a3fa5b7\" (UID: \"14c9cff3-3757-4daf-837e-cbb86a3fa5b7\") "
Sep 12 00:59:04.024156 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:04.024030    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zkq55\" (UniqueName: \"kubernetes.io/projected/14c9cff3-3757-4daf-837e-cbb86a3fa5b7-kube-api-access-zkq55\") pod \"14c9cff3-3757-4daf-837e-cbb86a3fa5b7\" (UID: \"14c9cff3-3757-4daf-837e-cbb86a3fa5b7\") "
Sep 12 00:59:04.024376 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:04.024350    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14c9cff3-3757-4daf-837e-cbb86a3fa5b7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "14c9cff3-3757-4daf-837e-cbb86a3fa5b7" (UID: "14c9cff3-3757-4daf-837e-cbb86a3fa5b7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:59:04.026390 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:04.026367    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14c9cff3-3757-4daf-837e-cbb86a3fa5b7-kube-api-access-zkq55" (OuterVolumeSpecName: "kube-api-access-zkq55") pod "14c9cff3-3757-4daf-837e-cbb86a3fa5b7" (UID: "14c9cff3-3757-4daf-837e-cbb86a3fa5b7"). InnerVolumeSpecName "kube-api-access-zkq55". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:59:04.050944 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:04.050920    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:04.125385 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:04.125336    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkq55\" (UniqueName: \"kubernetes.io/projected/14c9cff3-3757-4daf-837e-cbb86a3fa5b7-kube-api-access-zkq55\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:59:04.125385 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:04.125380    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14c9cff3-3757-4daf-837e-cbb86a3fa5b7-plugins-config-volume\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:59:04.170691 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:04.170656    2558 generic.go:356] "Generic (PLEG): container finished" podID="14c9cff3-3757-4daf-837e-cbb86a3fa5b7" containerID="aba16ac7a494a1fb8e6e2101396682aaa05802a844611d01823e822fe9d1dc05" exitCode=0
Sep 12 00:59:04.170836 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:04.170704    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-04f8e8-6fc7764877-9b8c6"
Sep 12 00:59:04.170836 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:04.170747    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-04f8e8-6fc7764877-9b8c6" event={"ID":"14c9cff3-3757-4daf-837e-cbb86a3fa5b7","Type":"ContainerDied","Data":"aba16ac7a494a1fb8e6e2101396682aaa05802a844611d01823e822fe9d1dc05"}
Sep 12 00:59:04.170836 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:04.170801    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-04f8e8-6fc7764877-9b8c6" event={"ID":"14c9cff3-3757-4daf-837e-cbb86a3fa5b7","Type":"ContainerDied","Data":"76204a370dbe366194958ddeb6a80e34cec0f66a2e99efc9e228987e3e706980"}
Sep 12 00:59:04.170836 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:04.170824    2558 scope.go:117] "RemoveContainer" containerID="aba16ac7a494a1fb8e6e2101396682aaa05802a844611d01823e822fe9d1dc05"
Sep 12 00:59:04.180580 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:04.180561    2558 scope.go:117] "RemoveContainer" containerID="aba16ac7a494a1fb8e6e2101396682aaa05802a844611d01823e822fe9d1dc05"
Sep 12 00:59:04.180865 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:59:04.180844    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aba16ac7a494a1fb8e6e2101396682aaa05802a844611d01823e822fe9d1dc05\": container with ID starting with aba16ac7a494a1fb8e6e2101396682aaa05802a844611d01823e822fe9d1dc05 not found: ID does not exist" containerID="aba16ac7a494a1fb8e6e2101396682aaa05802a844611d01823e822fe9d1dc05"
Sep 12 00:59:04.180917 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:04.180874    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aba16ac7a494a1fb8e6e2101396682aaa05802a844611d01823e822fe9d1dc05"} err="failed to get container status \"aba16ac7a494a1fb8e6e2101396682aaa05802a844611d01823e822fe9d1dc05\": rpc error: code = NotFound desc = could not find container \"aba16ac7a494a1fb8e6e2101396682aaa05802a844611d01823e822fe9d1dc05\": container with ID starting with aba16ac7a494a1fb8e6e2101396682aaa05802a844611d01823e822fe9d1dc05 not found: ID does not exist"
Sep 12 00:59:04.200648 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:04.200564    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-04f8e8-6fc7764877-9b8c6"]
Sep 12 00:59:04.203400 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:04.203374    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-04f8e8-6fc7764877-9b8c6"]
Sep 12 00:59:04.258330 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:04.258295    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:04.489077 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:04.488989    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:05.332021 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:05.331985    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:05.739029 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:05.738997    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14c9cff3-3757-4daf-837e-cbb86a3fa5b7" path="/var/lib/kubelet/pods/14c9cff3-3757-4daf-837e-cbb86a3fa5b7/volumes"
Sep 12 00:59:06.179261 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:06.179170    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:06.968520 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:06.968490    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:07.677417 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:07.677377    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:08.565237 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:08.565207    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:09.328189 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:09.328153    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:09.979275 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:09.979231    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:10.506461 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:10.506420    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:10.607430 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:10.607402    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:10.750418 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:10.750392    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:11.290696 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:11.290664    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:12.094659 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:12.094624    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:12.776018 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:12.775987    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:13.597114 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:13.597049    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:14.033099 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:14.033063    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:14.133678 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:14.133645    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:14.272444 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:14.272411    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:15.075568 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:15.075529    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:15.777068 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:15.777035    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:16.574138 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:16.574103    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:17.373674 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:17.373634    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:23.864735 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:23.864635    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-czf8h_dd90996a-5428-4ccf-a154-c0aa8def9789/console-operator/1.log"
Sep 12 00:59:23.882280 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:23.869039    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-czf8h_dd90996a-5428-4ccf-a154-c0aa8def9789/console-operator/1.log"
Sep 12 00:59:25.982368 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:25.982331    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:26.706688 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:26.706649    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c14a20-f885bd68-8krn2"]
Sep 12 00:59:26.706930 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:26.706898    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c14a20-f885bd68-8krn2" podUID="42381ab7-985d-4f04-ba78-ce3325c6bed8" containerName="maas-api" containerID="cri-o://5f5682d1a6e47d549114239a606d3b12e9f66487075d18c012fff61998a331c0" gracePeriod=30
Sep 12 00:59:26.734494 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:26.734463    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:26.796291 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:26.796264    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:26.897000 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:26.896956    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:26.957838 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:26.957778    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:26.968805 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:26.968783    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c14a20-f885bd68-8krn2"
Sep 12 00:59:27.065135 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:27.065106    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98zxg\" (UniqueName: \"kubernetes.io/projected/42381ab7-985d-4f04-ba78-ce3325c6bed8-kube-api-access-98zxg\") pod \"42381ab7-985d-4f04-ba78-ce3325c6bed8\" (UID: \"42381ab7-985d-4f04-ba78-ce3325c6bed8\") "
Sep 12 00:59:27.065576 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:27.065165    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42381ab7-985d-4f04-ba78-ce3325c6bed8-maas-api-tls\") pod \"42381ab7-985d-4f04-ba78-ce3325c6bed8\" (UID: \"42381ab7-985d-4f04-ba78-ce3325c6bed8\") "
Sep 12 00:59:27.065576 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:27.065211    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42381ab7-985d-4f04-ba78-ce3325c6bed8-metrics-certs\") pod \"42381ab7-985d-4f04-ba78-ce3325c6bed8\" (UID: \"42381ab7-985d-4f04-ba78-ce3325c6bed8\") "
Sep 12 00:59:27.065576 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:27.065281    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42381ab7-985d-4f04-ba78-ce3325c6bed8-openshift-service-ca\") pod \"42381ab7-985d-4f04-ba78-ce3325c6bed8\" (UID: \"42381ab7-985d-4f04-ba78-ce3325c6bed8\") "
Sep 12 00:59:27.065800 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:27.065768    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42381ab7-985d-4f04-ba78-ce3325c6bed8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "42381ab7-985d-4f04-ba78-ce3325c6bed8" (UID: "42381ab7-985d-4f04-ba78-ce3325c6bed8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:59:27.068452 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:27.068420    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42381ab7-985d-4f04-ba78-ce3325c6bed8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "42381ab7-985d-4f04-ba78-ce3325c6bed8" (UID: "42381ab7-985d-4f04-ba78-ce3325c6bed8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:59:27.068452 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:27.068444    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42381ab7-985d-4f04-ba78-ce3325c6bed8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "42381ab7-985d-4f04-ba78-ce3325c6bed8" (UID: "42381ab7-985d-4f04-ba78-ce3325c6bed8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:59:27.068611 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:27.068465    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42381ab7-985d-4f04-ba78-ce3325c6bed8-kube-api-access-98zxg" (OuterVolumeSpecName: "kube-api-access-98zxg") pod "42381ab7-985d-4f04-ba78-ce3325c6bed8" (UID: "42381ab7-985d-4f04-ba78-ce3325c6bed8"). InnerVolumeSpecName "kube-api-access-98zxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:59:27.166308 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:27.166274    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98zxg\" (UniqueName: \"kubernetes.io/projected/42381ab7-985d-4f04-ba78-ce3325c6bed8-kube-api-access-98zxg\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:59:27.166308 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:27.166301    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42381ab7-985d-4f04-ba78-ce3325c6bed8-maas-api-tls\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:59:27.166308 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:27.166310    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42381ab7-985d-4f04-ba78-ce3325c6bed8-metrics-certs\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:59:27.166308 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:27.166318    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42381ab7-985d-4f04-ba78-ce3325c6bed8-openshift-service-ca\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:59:27.256758 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:27.256663    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:27.296214 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:27.296180    2558 generic.go:356] "Generic (PLEG): container finished" podID="42381ab7-985d-4f04-ba78-ce3325c6bed8" containerID="5f5682d1a6e47d549114239a606d3b12e9f66487075d18c012fff61998a331c0" exitCode=0
Sep 12 00:59:27.296387 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:27.296255    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c14a20-f885bd68-8krn2"
Sep 12 00:59:27.296387 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:27.296264    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c14a20-f885bd68-8krn2" event={"ID":"42381ab7-985d-4f04-ba78-ce3325c6bed8","Type":"ContainerDied","Data":"5f5682d1a6e47d549114239a606d3b12e9f66487075d18c012fff61998a331c0"}
Sep 12 00:59:27.296387 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:27.296359    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c14a20-f885bd68-8krn2" event={"ID":"42381ab7-985d-4f04-ba78-ce3325c6bed8","Type":"ContainerDied","Data":"9e12cc1fe33d833f3f25b76249839cfda19b946076d98a25ba3a2dff46449cf6"}
Sep 12 00:59:27.296515 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:27.296399    2558 scope.go:117] "RemoveContainer" containerID="5f5682d1a6e47d549114239a606d3b12e9f66487075d18c012fff61998a331c0"
Sep 12 00:59:27.306242 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:27.306224    2558 scope.go:117] "RemoveContainer" containerID="5f5682d1a6e47d549114239a606d3b12e9f66487075d18c012fff61998a331c0"
Sep 12 00:59:27.306534 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:59:27.306517    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f5682d1a6e47d549114239a606d3b12e9f66487075d18c012fff61998a331c0\": container with ID starting with 5f5682d1a6e47d549114239a606d3b12e9f66487075d18c012fff61998a331c0 not found: ID does not exist" containerID="5f5682d1a6e47d549114239a606d3b12e9f66487075d18c012fff61998a331c0"
Sep 12 00:59:27.306576 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:27.306545    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f5682d1a6e47d549114239a606d3b12e9f66487075d18c012fff61998a331c0"} err="failed to get container status \"5f5682d1a6e47d549114239a606d3b12e9f66487075d18c012fff61998a331c0\": rpc error: code = NotFound desc = could not find container \"5f5682d1a6e47d549114239a606d3b12e9f66487075d18c012fff61998a331c0\": container with ID starting with 5f5682d1a6e47d549114239a606d3b12e9f66487075d18c012fff61998a331c0 not found: ID does not exist"
Sep 12 00:59:27.319342 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:27.319321    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c14a20-f885bd68-8krn2"]
Sep 12 00:59:27.322389 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:27.322366    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:27.323802 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:27.323783    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c14a20-f885bd68-8krn2"]
Sep 12 00:59:27.739877 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:27.739845    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42381ab7-985d-4f04-ba78-ce3325c6bed8" path="/var/lib/kubelet/pods/42381ab7-985d-4f04-ba78-ce3325c6bed8/volumes"
Sep 12 00:59:28.126613 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:28.126534    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:28.872644 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:28.872607    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:30.667722 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:30.667692    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:31.300555 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:31.300521    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:32.046164 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:32.046130    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:32.877314 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:32.877278    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:33.577471 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:33.577437    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:33.681265 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:33.681221    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:33.781763 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:33.781725    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:33.878178 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:33.878075    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:34.801697 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:34.801664    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:35.261409 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:35.261372    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw"]
Sep 12 00:59:35.266627 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:35.266602    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9"]
Sep 12 00:59:35.358665 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:35.358633    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:35.734309 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:35.734279    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:36.569314 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:36.569279    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:37.364961 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:37.364923    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:38.044045 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:38.044002    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:38.739498 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:38.739467    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:39.675266 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:39.675236    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:40.438219 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:40.438182    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:44.376668 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:44.376632    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:44.973809 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:44.973772    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:45.049043 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:45.048959    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:45.117378 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:45.117344    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:45.198882 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:45.198843    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:45.619318 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:45.619282    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:45.704182 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:45.704153    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:46.369641 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:46.369602    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:46.955878 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:46.955849    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:46.986014 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:46.985985    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-4d485e-65b7fbc5c9-llkv7"]
Sep 12 00:59:46.986299 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:46.986271    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-4d485e-65b7fbc5c9-llkv7" podUID="55a4d4f6-f077-4ade-9c75-f3c57e8abde4" containerName="payload-processing" containerID="cri-o://d4d581c8685af6a9d912f9567ceea5136d854140d751ac0a1cc9bb6638c0b47c" gracePeriod=30
Sep 12 00:59:46.991536 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:46.991485    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-4d485e-7c8bb46bcd-s9g6d"]
Sep 12 00:59:46.991724 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:46.991698    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d485e-7c8bb46bcd-s9g6d" podUID="325f7eb9-abb9-4f41-88b9-d488129d4024" containerName="payload-pre-processing" containerID="cri-o://51fad3a8d2bbfe2e15485ade9e38243549da315eefbb11fe4ce9e4939cb6f8fe" gracePeriod=30
Sep 12 00:59:47.041945 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.041914    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:47.124494 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.124465    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:47.224976 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.224941    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:47.258845 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.258810    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-4d485e-65b7fbc5c9-llkv7"
Sep 12 00:59:47.295286 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.295261    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d485e-7c8bb46bcd-s9g6d"
Sep 12 00:59:47.321270 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.321239    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:47.372122 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.372075    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fzx72\" (UniqueName: \"kubernetes.io/projected/55a4d4f6-f077-4ade-9c75-f3c57e8abde4-kube-api-access-fzx72\") pod \"55a4d4f6-f077-4ade-9c75-f3c57e8abde4\" (UID: \"55a4d4f6-f077-4ade-9c75-f3c57e8abde4\") "
Sep 12 00:59:47.372286 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.372168    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whrp8\" (UniqueName: \"kubernetes.io/projected/325f7eb9-abb9-4f41-88b9-d488129d4024-kube-api-access-whrp8\") pod \"325f7eb9-abb9-4f41-88b9-d488129d4024\" (UID: \"325f7eb9-abb9-4f41-88b9-d488129d4024\") "
Sep 12 00:59:47.372286 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.372224    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/325f7eb9-abb9-4f41-88b9-d488129d4024-plugins-config-volume\") pod \"325f7eb9-abb9-4f41-88b9-d488129d4024\" (UID: \"325f7eb9-abb9-4f41-88b9-d488129d4024\") "
Sep 12 00:59:47.372286 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.372252    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55a4d4f6-f077-4ade-9c75-f3c57e8abde4-plugins-config-volume\") pod \"55a4d4f6-f077-4ade-9c75-f3c57e8abde4\" (UID: \"55a4d4f6-f077-4ade-9c75-f3c57e8abde4\") "
Sep 12 00:59:47.372653 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.372625    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/325f7eb9-abb9-4f41-88b9-d488129d4024-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "325f7eb9-abb9-4f41-88b9-d488129d4024" (UID: "325f7eb9-abb9-4f41-88b9-d488129d4024"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:59:47.372653 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.372643    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55a4d4f6-f077-4ade-9c75-f3c57e8abde4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "55a4d4f6-f077-4ade-9c75-f3c57e8abde4" (UID: "55a4d4f6-f077-4ade-9c75-f3c57e8abde4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 00:59:47.374460 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.374438    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/325f7eb9-abb9-4f41-88b9-d488129d4024-kube-api-access-whrp8" (OuterVolumeSpecName: "kube-api-access-whrp8") pod "325f7eb9-abb9-4f41-88b9-d488129d4024" (UID: "325f7eb9-abb9-4f41-88b9-d488129d4024"). InnerVolumeSpecName "kube-api-access-whrp8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:59:47.374540 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.374463    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55a4d4f6-f077-4ade-9c75-f3c57e8abde4-kube-api-access-fzx72" (OuterVolumeSpecName: "kube-api-access-fzx72") pod "55a4d4f6-f077-4ade-9c75-f3c57e8abde4" (UID: "55a4d4f6-f077-4ade-9c75-f3c57e8abde4"). InnerVolumeSpecName "kube-api-access-fzx72". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:59:47.409657 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.409615    2558 generic.go:356] "Generic (PLEG): container finished" podID="55a4d4f6-f077-4ade-9c75-f3c57e8abde4" containerID="d4d581c8685af6a9d912f9567ceea5136d854140d751ac0a1cc9bb6638c0b47c" exitCode=0
Sep 12 00:59:47.409848 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.409674    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-4d485e-65b7fbc5c9-llkv7"
Sep 12 00:59:47.409848 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.409705    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-4d485e-65b7fbc5c9-llkv7" event={"ID":"55a4d4f6-f077-4ade-9c75-f3c57e8abde4","Type":"ContainerDied","Data":"d4d581c8685af6a9d912f9567ceea5136d854140d751ac0a1cc9bb6638c0b47c"}
Sep 12 00:59:47.409848 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.409770    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-4d485e-65b7fbc5c9-llkv7" event={"ID":"55a4d4f6-f077-4ade-9c75-f3c57e8abde4","Type":"ContainerDied","Data":"1e6ec3e8ebf4b5f64b8451438465b6e03182b4a9115c51b525a6a501f2b477cb"}
Sep 12 00:59:47.409848 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.409803    2558 scope.go:117] "RemoveContainer" containerID="d4d581c8685af6a9d912f9567ceea5136d854140d751ac0a1cc9bb6638c0b47c"
Sep 12 00:59:47.411778 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.411741    2558 generic.go:356] "Generic (PLEG): container finished" podID="325f7eb9-abb9-4f41-88b9-d488129d4024" containerID="51fad3a8d2bbfe2e15485ade9e38243549da315eefbb11fe4ce9e4939cb6f8fe" exitCode=0
Sep 12 00:59:47.411891 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.411795    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d485e-7c8bb46bcd-s9g6d"
Sep 12 00:59:47.411891 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.411824    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d485e-7c8bb46bcd-s9g6d" event={"ID":"325f7eb9-abb9-4f41-88b9-d488129d4024","Type":"ContainerDied","Data":"51fad3a8d2bbfe2e15485ade9e38243549da315eefbb11fe4ce9e4939cb6f8fe"}
Sep 12 00:59:47.411891 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.411863    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4d485e-7c8bb46bcd-s9g6d" event={"ID":"325f7eb9-abb9-4f41-88b9-d488129d4024","Type":"ContainerDied","Data":"ba8899d991a9328ea52efbc2de0ee341a3df6932ff21725e6e5f864fd1fc988f"}
Sep 12 00:59:47.422747 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.422722    2558 scope.go:117] "RemoveContainer" containerID="d4d581c8685af6a9d912f9567ceea5136d854140d751ac0a1cc9bb6638c0b47c"
Sep 12 00:59:47.423191 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:59:47.423168    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4d581c8685af6a9d912f9567ceea5136d854140d751ac0a1cc9bb6638c0b47c\": container with ID starting with d4d581c8685af6a9d912f9567ceea5136d854140d751ac0a1cc9bb6638c0b47c not found: ID does not exist" containerID="d4d581c8685af6a9d912f9567ceea5136d854140d751ac0a1cc9bb6638c0b47c"
Sep 12 00:59:47.423297 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.423199    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4d581c8685af6a9d912f9567ceea5136d854140d751ac0a1cc9bb6638c0b47c"} err="failed to get container status \"d4d581c8685af6a9d912f9567ceea5136d854140d751ac0a1cc9bb6638c0b47c\": rpc error: code = NotFound desc = could not find container \"d4d581c8685af6a9d912f9567ceea5136d854140d751ac0a1cc9bb6638c0b47c\": container with ID starting with d4d581c8685af6a9d912f9567ceea5136d854140d751ac0a1cc9bb6638c0b47c not found: ID does not exist"
Sep 12 00:59:47.423297 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.423216    2558 scope.go:117] "RemoveContainer" containerID="51fad3a8d2bbfe2e15485ade9e38243549da315eefbb11fe4ce9e4939cb6f8fe"
Sep 12 00:59:47.434150 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.434113    2558 scope.go:117] "RemoveContainer" containerID="51fad3a8d2bbfe2e15485ade9e38243549da315eefbb11fe4ce9e4939cb6f8fe"
Sep 12 00:59:47.434439 ip-10-0-142-85 kubenswrapper[2558]: E0912 00:59:47.434420    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51fad3a8d2bbfe2e15485ade9e38243549da315eefbb11fe4ce9e4939cb6f8fe\": container with ID starting with 51fad3a8d2bbfe2e15485ade9e38243549da315eefbb11fe4ce9e4939cb6f8fe not found: ID does not exist" containerID="51fad3a8d2bbfe2e15485ade9e38243549da315eefbb11fe4ce9e4939cb6f8fe"
Sep 12 00:59:47.434545 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.434446    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51fad3a8d2bbfe2e15485ade9e38243549da315eefbb11fe4ce9e4939cb6f8fe"} err="failed to get container status \"51fad3a8d2bbfe2e15485ade9e38243549da315eefbb11fe4ce9e4939cb6f8fe\": rpc error: code = NotFound desc = could not find container \"51fad3a8d2bbfe2e15485ade9e38243549da315eefbb11fe4ce9e4939cb6f8fe\": container with ID starting with 51fad3a8d2bbfe2e15485ade9e38243549da315eefbb11fe4ce9e4939cb6f8fe not found: ID does not exist"
Sep 12 00:59:47.436281 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.436258    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-4d485e-65b7fbc5c9-llkv7"]
Sep 12 00:59:47.439899 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.439879    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-4d485e-65b7fbc5c9-llkv7"]
Sep 12 00:59:47.452050 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.452027    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-4d485e-7c8bb46bcd-s9g6d"]
Sep 12 00:59:47.455796 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.455775    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-4d485e-7c8bb46bcd-s9g6d"]
Sep 12 00:59:47.466985 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.466901    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:47.473250 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.473223    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzx72\" (UniqueName: \"kubernetes.io/projected/55a4d4f6-f077-4ade-9c75-f3c57e8abde4-kube-api-access-fzx72\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:59:47.473250 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.473246    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whrp8\" (UniqueName: \"kubernetes.io/projected/325f7eb9-abb9-4f41-88b9-d488129d4024-kube-api-access-whrp8\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:59:47.473389 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.473256    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/325f7eb9-abb9-4f41-88b9-d488129d4024-plugins-config-volume\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:59:47.473389 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.473264    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55a4d4f6-f077-4ade-9c75-f3c57e8abde4-plugins-config-volume\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:59:47.676884 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.676840    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:47.738837 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.738755    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="325f7eb9-abb9-4f41-88b9-d488129d4024" path="/var/lib/kubelet/pods/325f7eb9-abb9-4f41-88b9-d488129d4024/volumes"
Sep 12 00:59:47.739172 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:47.739159    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55a4d4f6-f077-4ade-9c75-f3c57e8abde4" path="/var/lib/kubelet/pods/55a4d4f6-f077-4ade-9c75-f3c57e8abde4/volumes"
Sep 12 00:59:48.397522 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:48.397423    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:49.149205 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:49.149165    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:49.983589 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:49.983544    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:50.281998 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.281905    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw" podUID="a19c70cb-f6f2-4f28-b05b-eeeda341a3c8" containerName="main" containerID="cri-o://ebf24c63459e63436b4e8d1c952dc513cea20b5eb74c63172cd328a79d054d89" gracePeriod=45
Sep 12 00:59:50.286049 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.286012    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9" podUID="78e29efa-f066-4fc5-a761-89dcf4ecb7a1" containerName="main" containerID="cri-o://fff6499613415564ec7b885e9d5e0ad9e3bebb18628c4902f8d9ea503991ab27" gracePeriod=45
Sep 12 00:59:50.432311 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.432269    2558 generic.go:356] "Generic (PLEG): container finished" podID="a19c70cb-f6f2-4f28-b05b-eeeda341a3c8" containerID="ebf24c63459e63436b4e8d1c952dc513cea20b5eb74c63172cd328a79d054d89" exitCode=0
Sep 12 00:59:50.432506 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.432348    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw" event={"ID":"a19c70cb-f6f2-4f28-b05b-eeeda341a3c8","Type":"ContainerDied","Data":"ebf24c63459e63436b4e8d1c952dc513cea20b5eb74c63172cd328a79d054d89"}
Sep 12 00:59:50.434295 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.434269    2558 generic.go:356] "Generic (PLEG): container finished" podID="78e29efa-f066-4fc5-a761-89dcf4ecb7a1" containerID="fff6499613415564ec7b885e9d5e0ad9e3bebb18628c4902f8d9ea503991ab27" exitCode=0
Sep 12 00:59:50.434451 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.434340    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9" event={"ID":"78e29efa-f066-4fc5-a761-89dcf4ecb7a1","Type":"ContainerDied","Data":"fff6499613415564ec7b885e9d5e0ad9e3bebb18628c4902f8d9ea503991ab27"}
Sep 12 00:59:50.577288 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.577068    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw"
Sep 12 00:59:50.581247 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.581213    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9"
Sep 12 00:59:50.705732 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.705697    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:50.715840 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.715807    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-tmp-dir\") pod \"78e29efa-f066-4fc5-a761-89dcf4ecb7a1\" (UID: \"78e29efa-f066-4fc5-a761-89dcf4ecb7a1\") "
Sep 12 00:59:50.716018 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.715898    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-dshm\") pod \"78e29efa-f066-4fc5-a761-89dcf4ecb7a1\" (UID: \"78e29efa-f066-4fc5-a761-89dcf4ecb7a1\") "
Sep 12 00:59:50.716018 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.715947    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-tls-certs\") pod \"78e29efa-f066-4fc5-a761-89dcf4ecb7a1\" (UID: \"78e29efa-f066-4fc5-a761-89dcf4ecb7a1\") "
Sep 12 00:59:50.716018 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.715975    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-tls-certs\") pod \"a19c70cb-f6f2-4f28-b05b-eeeda341a3c8\" (UID: \"a19c70cb-f6f2-4f28-b05b-eeeda341a3c8\") "
Sep 12 00:59:50.716283 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.716034    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-model-cache\") pod \"a19c70cb-f6f2-4f28-b05b-eeeda341a3c8\" (UID: \"a19c70cb-f6f2-4f28-b05b-eeeda341a3c8\") "
Sep 12 00:59:50.716283 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.716036    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "78e29efa-f066-4fc5-a761-89dcf4ecb7a1" (UID: "78e29efa-f066-4fc5-a761-89dcf4ecb7a1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:59:50.716283 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.716063    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2497j\" (UniqueName: \"kubernetes.io/projected/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-kube-api-access-2497j\") pod \"a19c70cb-f6f2-4f28-b05b-eeeda341a3c8\" (UID: \"a19c70cb-f6f2-4f28-b05b-eeeda341a3c8\") "
Sep 12 00:59:50.716283 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.716127    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-tmp-dir\") pod \"a19c70cb-f6f2-4f28-b05b-eeeda341a3c8\" (UID: \"a19c70cb-f6f2-4f28-b05b-eeeda341a3c8\") "
Sep 12 00:59:50.716283 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.716187    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kj9t7\" (UniqueName: \"kubernetes.io/projected/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-kube-api-access-kj9t7\") pod \"78e29efa-f066-4fc5-a761-89dcf4ecb7a1\" (UID: \"78e29efa-f066-4fc5-a761-89dcf4ecb7a1\") "
Sep 12 00:59:50.716283 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.716213    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-home\") pod \"a19c70cb-f6f2-4f28-b05b-eeeda341a3c8\" (UID: \"a19c70cb-f6f2-4f28-b05b-eeeda341a3c8\") "
Sep 12 00:59:50.716573 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.716312    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-model-cache\") pod \"78e29efa-f066-4fc5-a761-89dcf4ecb7a1\" (UID: \"78e29efa-f066-4fc5-a761-89dcf4ecb7a1\") "
Sep 12 00:59:50.716573 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.716335    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-dshm\") pod \"a19c70cb-f6f2-4f28-b05b-eeeda341a3c8\" (UID: \"a19c70cb-f6f2-4f28-b05b-eeeda341a3c8\") "
Sep 12 00:59:50.716573 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.716359    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-home\") pod \"78e29efa-f066-4fc5-a761-89dcf4ecb7a1\" (UID: \"78e29efa-f066-4fc5-a761-89dcf4ecb7a1\") "
Sep 12 00:59:50.716573 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.716491    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-model-cache" (OuterVolumeSpecName: "model-cache") pod "a19c70cb-f6f2-4f28-b05b-eeeda341a3c8" (UID: "a19c70cb-f6f2-4f28-b05b-eeeda341a3c8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:59:50.716772 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.716710    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a19c70cb-f6f2-4f28-b05b-eeeda341a3c8" (UID: "a19c70cb-f6f2-4f28-b05b-eeeda341a3c8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:59:50.716981 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.716860    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-model-cache\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:59:50.716981 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.716871    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-model-cache" (OuterVolumeSpecName: "model-cache") pod "78e29efa-f066-4fc5-a761-89dcf4ecb7a1" (UID: "78e29efa-f066-4fc5-a761-89dcf4ecb7a1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:59:50.716981 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.716886    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-tmp-dir\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:59:50.716981 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.716899    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-tmp-dir\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:59:50.717258 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.717013    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-home" (OuterVolumeSpecName: "home") pod "a19c70cb-f6f2-4f28-b05b-eeeda341a3c8" (UID: "a19c70cb-f6f2-4f28-b05b-eeeda341a3c8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:59:50.717258 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.717077    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-home" (OuterVolumeSpecName: "home") pod "78e29efa-f066-4fc5-a761-89dcf4ecb7a1" (UID: "78e29efa-f066-4fc5-a761-89dcf4ecb7a1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:59:50.718909 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.718879    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "78e29efa-f066-4fc5-a761-89dcf4ecb7a1" (UID: "78e29efa-f066-4fc5-a761-89dcf4ecb7a1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:59:50.719474 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.719421    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a19c70cb-f6f2-4f28-b05b-eeeda341a3c8" (UID: "a19c70cb-f6f2-4f28-b05b-eeeda341a3c8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 00:59:50.719474 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.719424    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-kube-api-access-2497j" (OuterVolumeSpecName: "kube-api-access-2497j") pod "a19c70cb-f6f2-4f28-b05b-eeeda341a3c8" (UID: "a19c70cb-f6f2-4f28-b05b-eeeda341a3c8"). InnerVolumeSpecName "kube-api-access-2497j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:59:50.719662 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.719591    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-dshm" (OuterVolumeSpecName: "dshm") pod "a19c70cb-f6f2-4f28-b05b-eeeda341a3c8" (UID: "a19c70cb-f6f2-4f28-b05b-eeeda341a3c8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:59:50.719893 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.719865    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-dshm" (OuterVolumeSpecName: "dshm") pod "78e29efa-f066-4fc5-a761-89dcf4ecb7a1" (UID: "78e29efa-f066-4fc5-a761-89dcf4ecb7a1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 00:59:50.720246 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.720218    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-kube-api-access-kj9t7" (OuterVolumeSpecName: "kube-api-access-kj9t7") pod "78e29efa-f066-4fc5-a761-89dcf4ecb7a1" (UID: "78e29efa-f066-4fc5-a761-89dcf4ecb7a1"). InnerVolumeSpecName "kube-api-access-kj9t7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 00:59:50.817809 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.817580    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-dshm\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:59:50.817809 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.817620    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-tls-certs\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:59:50.817809 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.817634    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-tls-certs\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:59:50.817809 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.817646    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2497j\" (UniqueName: \"kubernetes.io/projected/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-kube-api-access-2497j\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:59:50.817809 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.817659    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kj9t7\" (UniqueName: \"kubernetes.io/projected/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-kube-api-access-kj9t7\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:59:50.817809 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.817672    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-home\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:59:50.817809 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.817685    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-model-cache\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:59:50.817809 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.817697    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8-dshm\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:59:50.817809 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:50.817709    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78e29efa-f066-4fc5-a761-89dcf4ecb7a1-home\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 00:59:51.429652 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:51.429609    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:51.441317 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:51.441295    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9"
Sep 12 00:59:51.441465 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:51.441337    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9" event={"ID":"78e29efa-f066-4fc5-a761-89dcf4ecb7a1","Type":"ContainerDied","Data":"f23fa8a135b6977e0ecd690c580a3a026dbcce9e6c2d21973933a781a69824e5"}
Sep 12 00:59:51.441465 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:51.441393    2558 scope.go:117] "RemoveContainer" containerID="fff6499613415564ec7b885e9d5e0ad9e3bebb18628c4902f8d9ea503991ab27"
Sep 12 00:59:51.442834 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:51.442798    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw"
Sep 12 00:59:51.442954 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:51.442842    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw" event={"ID":"a19c70cb-f6f2-4f28-b05b-eeeda341a3c8","Type":"ContainerDied","Data":"1edccae8600b819264ecbb83cdfcfcca224bce84818d7cb145e8f67c1a00e7d1"}
Sep 12 00:59:51.452711 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:51.452660    2558 scope.go:117] "RemoveContainer" containerID="ebf24c63459e63436b4e8d1c952dc513cea20b5eb74c63172cd328a79d054d89"
Sep 12 00:59:51.463633 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:51.463612    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9"]
Sep 12 00:59:51.469723 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:51.469704    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-4d485e/distinct-w0-4d485e-kserve-9b7f797d5-l5fg9"]
Sep 12 00:59:51.479328 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:51.479306    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw"]
Sep 12 00:59:51.483262 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:51.483237    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-4d485e/distinct-2-w0-4d485e-kserve-544cc565c4-gknqw"]
Sep 12 00:59:51.739265 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:51.739229    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78e29efa-f066-4fc5-a761-89dcf4ecb7a1" path="/var/lib/kubelet/pods/78e29efa-f066-4fc5-a761-89dcf4ecb7a1/volumes"
Sep 12 00:59:51.739681 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:51.739667    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a19c70cb-f6f2-4f28-b05b-eeeda341a3c8" path="/var/lib/kubelet/pods/a19c70cb-f6f2-4f28-b05b-eeeda341a3c8/volumes"
Sep 12 00:59:51.960214 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:51.960184    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:52.059535 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:52.059452    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:52.304235 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:52.304204    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:52.848306 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:52.848270    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:53.597078 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:53.597048    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:54.245344 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:54.245303    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:54.345691 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:54.345657    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:54.512238 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:54.512149    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:54.599663 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:54.599623    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:55.288831 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:55.288796    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:56.068889 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:56.068853    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 00:59:56.737312 ip-10-0-142-85 kubenswrapper[2558]: I0912 00:59:56.737277    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:00.141603 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:00.141567    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-dd25dd-29819580-qxrx9"]
Sep 12 01:00:00.141993 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:00.141976    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a19c70cb-f6f2-4f28-b05b-eeeda341a3c8" containerName="main"
Sep 12 01:00:00.141993 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:00.141987    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a19c70cb-f6f2-4f28-b05b-eeeda341a3c8" containerName="main"
Sep 12 01:00:00.142067 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:00.142000    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14c9cff3-3757-4daf-837e-cbb86a3fa5b7" containerName="payload-processing"
Sep 12 01:00:00.142067 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:00.142006    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="14c9cff3-3757-4daf-837e-cbb86a3fa5b7" containerName="payload-processing"
Sep 12 01:00:00.142067 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:00.142016    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42381ab7-985d-4f04-ba78-ce3325c6bed8" containerName="maas-api"
Sep 12 01:00:00.142067 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:00.142021    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="42381ab7-985d-4f04-ba78-ce3325c6bed8" containerName="maas-api"
Sep 12 01:00:00.142067 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:00.142034    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="325f7eb9-abb9-4f41-88b9-d488129d4024" containerName="payload-pre-processing"
Sep 12 01:00:00.142067 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:00.142039    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="325f7eb9-abb9-4f41-88b9-d488129d4024" containerName="payload-pre-processing"
Sep 12 01:00:00.142067 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:00.142045    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55a4d4f6-f077-4ade-9c75-f3c57e8abde4" containerName="payload-processing"
Sep 12 01:00:00.142067 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:00.142050    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="55a4d4f6-f077-4ade-9c75-f3c57e8abde4" containerName="payload-processing"
Sep 12 01:00:00.142067 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:00.142060    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78e29efa-f066-4fc5-a761-89dcf4ecb7a1" containerName="main"
Sep 12 01:00:00.142067 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:00.142065    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="78e29efa-f066-4fc5-a761-89dcf4ecb7a1" containerName="main"
Sep 12 01:00:00.142386 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:00.142163    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="14c9cff3-3757-4daf-837e-cbb86a3fa5b7" containerName="payload-processing"
Sep 12 01:00:00.142386 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:00.142172    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="78e29efa-f066-4fc5-a761-89dcf4ecb7a1" containerName="main"
Sep 12 01:00:00.142386 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:00.142178    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="55a4d4f6-f077-4ade-9c75-f3c57e8abde4" containerName="payload-processing"
Sep 12 01:00:00.142386 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:00.142184    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a19c70cb-f6f2-4f28-b05b-eeeda341a3c8" containerName="main"
Sep 12 01:00:00.142386 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:00.142191    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="325f7eb9-abb9-4f41-88b9-d488129d4024" containerName="payload-pre-processing"
Sep 12 01:00:00.142386 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:00.142197    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="42381ab7-985d-4f04-ba78-ce3325c6bed8" containerName="maas-api"
Sep 12 01:00:00.157499 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:00.157470    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-dd25dd-29819580-qxrx9"]
Sep 12 01:00:00.157683 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:00.157607    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-dd25dd-29819580-qxrx9"
Sep 12 01:00:00.160281 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:00.160257    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sj5jt\""
Sep 12 01:00:00.168037 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:00.168013    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-dd25dd-29819580-qxrx9"
Sep 12 01:00:00.392310 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:00.392223    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:00.446109 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:00.443505    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-dd25dd-29819580-qxrx9"]
Sep 12 01:00:00.492938 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:00.492898    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-dd25dd-29819580-qxrx9" event={"ID":"a7279318-9146-4ec3-8f17-b7994c21f31f","Type":"ContainerStarted","Data":"8f3b94c2c59d45725c99079e7840d348396470ea815a91d888ec952485babe2f"}
Sep 12 01:00:02.800144 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:02.800112    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:03.399970 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:03.399883    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:03.511695 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:03.511652    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-dd25dd-29819580-qxrx9" event={"ID":"a7279318-9146-4ec3-8f17-b7994c21f31f","Type":"ContainerStarted","Data":"1e782de04198789a5aa408a8664f292e8e075adb51a7ee19c77728a965ac0c1c"}
Sep 12 01:00:03.529306 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:03.529256    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-dd25dd-29819580-qxrx9" podStartSLOduration=1.115369106 podStartE2EDuration="3.529239546s" totalImagesPullingTime="2.41387044s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:00:03.527158209 +0000 UTC m=+1840.201217068" watchObservedRunningTime="2026-09-12 01:00:03.529239546 +0000 UTC m=+1840.203298401"
Sep 12 01:00:03.708080 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:03.708045    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:03.779499 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:03.779460    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:03.921053 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:03.921026    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:04.019786 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:04.019700    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:04.519488 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:04.519452    2558 generic.go:356] "Generic (PLEG): container finished" podID="a7279318-9146-4ec3-8f17-b7994c21f31f" containerID="1e782de04198789a5aa408a8664f292e8e075adb51a7ee19c77728a965ac0c1c" exitCode=0
Sep 12 01:00:04.519671 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:04.519558    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-dd25dd-29819580-qxrx9" event={"ID":"a7279318-9146-4ec3-8f17-b7994c21f31f","Type":"ContainerDied","Data":"1e782de04198789a5aa408a8664f292e8e075adb51a7ee19c77728a965ac0c1c"}
Sep 12 01:00:04.814458 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:04.814365    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:05.662852 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:05.662823    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-dd25dd-29819580-qxrx9"
Sep 12 01:00:05.773930 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:05.773899    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:06.477231 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:06.477194    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:06.532721 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:06.532688    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-dd25dd-29819580-qxrx9" event={"ID":"a7279318-9146-4ec3-8f17-b7994c21f31f","Type":"ContainerDied","Data":"8f3b94c2c59d45725c99079e7840d348396470ea815a91d888ec952485babe2f"}
Sep 12 01:00:06.532893 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:06.532728    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-dd25dd-29819580-qxrx9"
Sep 12 01:00:06.532893 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:06.532745    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8f3b94c2c59d45725c99079e7840d348396470ea815a91d888ec952485babe2f"
Sep 12 01:00:09.854730 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:09.854694    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:09.963424 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:09.963385    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:10.144944 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:10.144836    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:11.764923 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:11.764871    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:11.865600 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:11.865571    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:13.081634 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:13.081597    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:18.608657 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:18.608622    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:23.281061 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:23.281025    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:23.731174 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:23.731137    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:24.432922 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:24.432822    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:25.297784 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:25.297747    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:25.971914 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:25.971878    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:26.961991 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:26.961951    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:28.261047 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:28.260997    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk"]
Sep 12 01:00:28.336956 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:28.336921    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:38.938726 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:38.938694    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb"]
Sep 12 01:00:38.939285 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:38.938969    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb" podUID="56fdc776-1e9f-4d9d-ac51-536cc781215e" containerName="maas-api" containerID="cri-o://5d9a83395a585f6064824d61e28737904d865cc199ac9b03277b01816c7bd12b" gracePeriod=30
Sep 12 01:00:38.984549 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:38.984514    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:38.997558 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:38.997521    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-876808-6494cfc7b9-xwsw7"]
Sep 12 01:00:38.997839 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:38.997790    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-876808-6494cfc7b9-xwsw7" podUID="de5f8abf-8e88-43a0-9f50-e1785612895f" containerName="payload-processing" containerID="cri-o://28da67938bcca5f6195c267cc5920aa5841f72a7e98769cf4d000b3966a164bb" gracePeriod=30
Sep 12 01:00:39.003764 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.003734    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-876808-7c65579d9d-sqzzc"]
Sep 12 01:00:39.004017 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.003953    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-876808-7c65579d9d-sqzzc" podUID="49a730e9-c6fe-4d17-bd64-233e1d3ba62a" containerName="payload-pre-processing" containerID="cri-o://2840385bc6469773a23866db5c89163467724ab8dda017eeb021438e0b8dec61" gracePeriod=30
Sep 12 01:00:39.085474 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.085442    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:39.175984 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.175938    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:39.238476 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.238446    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:39.271931 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.271886    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb"
Sep 12 01:00:39.276767 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.276740    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-876808-6494cfc7b9-xwsw7"
Sep 12 01:00:39.317553 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.317528    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-876808-7c65579d9d-sqzzc"
Sep 12 01:00:39.419021 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.418985    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7sj2\" (UniqueName: \"kubernetes.io/projected/49a730e9-c6fe-4d17-bd64-233e1d3ba62a-kube-api-access-x7sj2\") pod \"49a730e9-c6fe-4d17-bd64-233e1d3ba62a\" (UID: \"49a730e9-c6fe-4d17-bd64-233e1d3ba62a\") "
Sep 12 01:00:39.419248 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.419046    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56fdc776-1e9f-4d9d-ac51-536cc781215e-openshift-service-ca\") pod \"56fdc776-1e9f-4d9d-ac51-536cc781215e\" (UID: \"56fdc776-1e9f-4d9d-ac51-536cc781215e\") "
Sep 12 01:00:39.419248 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.419114    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xvvp9\" (UniqueName: \"kubernetes.io/projected/56fdc776-1e9f-4d9d-ac51-536cc781215e-kube-api-access-xvvp9\") pod \"56fdc776-1e9f-4d9d-ac51-536cc781215e\" (UID: \"56fdc776-1e9f-4d9d-ac51-536cc781215e\") "
Sep 12 01:00:39.419248 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.419144    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56fdc776-1e9f-4d9d-ac51-536cc781215e-maas-api-tls\") pod \"56fdc776-1e9f-4d9d-ac51-536cc781215e\" (UID: \"56fdc776-1e9f-4d9d-ac51-536cc781215e\") "
Sep 12 01:00:39.419248 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.419213    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56fdc776-1e9f-4d9d-ac51-536cc781215e-metrics-certs\") pod \"56fdc776-1e9f-4d9d-ac51-536cc781215e\" (UID: \"56fdc776-1e9f-4d9d-ac51-536cc781215e\") "
Sep 12 01:00:39.419248 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.419234    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qzzj\" (UniqueName: \"kubernetes.io/projected/de5f8abf-8e88-43a0-9f50-e1785612895f-kube-api-access-6qzzj\") pod \"de5f8abf-8e88-43a0-9f50-e1785612895f\" (UID: \"de5f8abf-8e88-43a0-9f50-e1785612895f\") "
Sep 12 01:00:39.419503 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.419260    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49a730e9-c6fe-4d17-bd64-233e1d3ba62a-plugins-config-volume\") pod \"49a730e9-c6fe-4d17-bd64-233e1d3ba62a\" (UID: \"49a730e9-c6fe-4d17-bd64-233e1d3ba62a\") "
Sep 12 01:00:39.419503 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.419285    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de5f8abf-8e88-43a0-9f50-e1785612895f-plugins-config-volume\") pod \"de5f8abf-8e88-43a0-9f50-e1785612895f\" (UID: \"de5f8abf-8e88-43a0-9f50-e1785612895f\") "
Sep 12 01:00:39.419503 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.419383    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56fdc776-1e9f-4d9d-ac51-536cc781215e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "56fdc776-1e9f-4d9d-ac51-536cc781215e" (UID: "56fdc776-1e9f-4d9d-ac51-536cc781215e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:00:39.419739 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.419704    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56fdc776-1e9f-4d9d-ac51-536cc781215e-openshift-service-ca\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:00:39.419739 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.419697    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49a730e9-c6fe-4d17-bd64-233e1d3ba62a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "49a730e9-c6fe-4d17-bd64-233e1d3ba62a" (UID: "49a730e9-c6fe-4d17-bd64-233e1d3ba62a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:00:39.419988 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.419945    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de5f8abf-8e88-43a0-9f50-e1785612895f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "de5f8abf-8e88-43a0-9f50-e1785612895f" (UID: "de5f8abf-8e88-43a0-9f50-e1785612895f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:00:39.421630 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.421600    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de5f8abf-8e88-43a0-9f50-e1785612895f-kube-api-access-6qzzj" (OuterVolumeSpecName: "kube-api-access-6qzzj") pod "de5f8abf-8e88-43a0-9f50-e1785612895f" (UID: "de5f8abf-8e88-43a0-9f50-e1785612895f"). InnerVolumeSpecName "kube-api-access-6qzzj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:00:39.421887 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.421870    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49a730e9-c6fe-4d17-bd64-233e1d3ba62a-kube-api-access-x7sj2" (OuterVolumeSpecName: "kube-api-access-x7sj2") pod "49a730e9-c6fe-4d17-bd64-233e1d3ba62a" (UID: "49a730e9-c6fe-4d17-bd64-233e1d3ba62a"). InnerVolumeSpecName "kube-api-access-x7sj2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:00:39.421940 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.421893    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56fdc776-1e9f-4d9d-ac51-536cc781215e-kube-api-access-xvvp9" (OuterVolumeSpecName: "kube-api-access-xvvp9") pod "56fdc776-1e9f-4d9d-ac51-536cc781215e" (UID: "56fdc776-1e9f-4d9d-ac51-536cc781215e"). InnerVolumeSpecName "kube-api-access-xvvp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:00:39.422135 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.422112    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56fdc776-1e9f-4d9d-ac51-536cc781215e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "56fdc776-1e9f-4d9d-ac51-536cc781215e" (UID: "56fdc776-1e9f-4d9d-ac51-536cc781215e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 01:00:39.422238 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.422212    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56fdc776-1e9f-4d9d-ac51-536cc781215e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "56fdc776-1e9f-4d9d-ac51-536cc781215e" (UID: "56fdc776-1e9f-4d9d-ac51-536cc781215e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 01:00:39.521037 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.520939    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7sj2\" (UniqueName: \"kubernetes.io/projected/49a730e9-c6fe-4d17-bd64-233e1d3ba62a-kube-api-access-x7sj2\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:00:39.521037 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.520973    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvvp9\" (UniqueName: \"kubernetes.io/projected/56fdc776-1e9f-4d9d-ac51-536cc781215e-kube-api-access-xvvp9\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:00:39.521037 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.520984    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56fdc776-1e9f-4d9d-ac51-536cc781215e-maas-api-tls\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:00:39.521037 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.520993    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56fdc776-1e9f-4d9d-ac51-536cc781215e-metrics-certs\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:00:39.521037 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.521001    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qzzj\" (UniqueName: \"kubernetes.io/projected/de5f8abf-8e88-43a0-9f50-e1785612895f-kube-api-access-6qzzj\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:00:39.521037 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.521011    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49a730e9-c6fe-4d17-bd64-233e1d3ba62a-plugins-config-volume\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:00:39.521037 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.521024    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de5f8abf-8e88-43a0-9f50-e1785612895f-plugins-config-volume\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:00:39.522694 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.522668    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:39.623856 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.623822    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:39.708020 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.707990    2558 generic.go:356] "Generic (PLEG): container finished" podID="de5f8abf-8e88-43a0-9f50-e1785612895f" containerID="28da67938bcca5f6195c267cc5920aa5841f72a7e98769cf4d000b3966a164bb" exitCode=0
Sep 12 01:00:39.708237 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.708027    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-876808-6494cfc7b9-xwsw7"
Sep 12 01:00:39.708237 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.708071    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-876808-6494cfc7b9-xwsw7" event={"ID":"de5f8abf-8e88-43a0-9f50-e1785612895f","Type":"ContainerDied","Data":"28da67938bcca5f6195c267cc5920aa5841f72a7e98769cf4d000b3966a164bb"}
Sep 12 01:00:39.708237 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.708136    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-876808-6494cfc7b9-xwsw7" event={"ID":"de5f8abf-8e88-43a0-9f50-e1785612895f","Type":"ContainerDied","Data":"a6ec029e7d250c6f3098c00b660fe5067b82df839d794457313aa051d99ea919"}
Sep 12 01:00:39.708237 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.708162    2558 scope.go:117] "RemoveContainer" containerID="28da67938bcca5f6195c267cc5920aa5841f72a7e98769cf4d000b3966a164bb"
Sep 12 01:00:39.709554 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.709532    2558 generic.go:356] "Generic (PLEG): container finished" podID="49a730e9-c6fe-4d17-bd64-233e1d3ba62a" containerID="2840385bc6469773a23866db5c89163467724ab8dda017eeb021438e0b8dec61" exitCode=0
Sep 12 01:00:39.709678 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.709613    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-876808-7c65579d9d-sqzzc"
Sep 12 01:00:39.709755 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.709609    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-876808-7c65579d9d-sqzzc" event={"ID":"49a730e9-c6fe-4d17-bd64-233e1d3ba62a","Type":"ContainerDied","Data":"2840385bc6469773a23866db5c89163467724ab8dda017eeb021438e0b8dec61"}
Sep 12 01:00:39.709811 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.709793    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-876808-7c65579d9d-sqzzc" event={"ID":"49a730e9-c6fe-4d17-bd64-233e1d3ba62a","Type":"ContainerDied","Data":"73f5af4190fb9813d09a4071e07ed7ebec568821e88a9ef8176a2a4a4a6afb32"}
Sep 12 01:00:39.711094 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.711064    2558 generic.go:356] "Generic (PLEG): container finished" podID="56fdc776-1e9f-4d9d-ac51-536cc781215e" containerID="5d9a83395a585f6064824d61e28737904d865cc199ac9b03277b01816c7bd12b" exitCode=0
Sep 12 01:00:39.711239 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.711127    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb" event={"ID":"56fdc776-1e9f-4d9d-ac51-536cc781215e","Type":"ContainerDied","Data":"5d9a83395a585f6064824d61e28737904d865cc199ac9b03277b01816c7bd12b"}
Sep 12 01:00:39.711239 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.711223    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb" event={"ID":"56fdc776-1e9f-4d9d-ac51-536cc781215e","Type":"ContainerDied","Data":"e148fe4f6215a3689c0804a38236c2343b59d4c4debd933f01a52aaf18042e58"}
Sep 12 01:00:39.711239 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.711133    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb"
Sep 12 01:00:39.718770 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.718751    2558 scope.go:117] "RemoveContainer" containerID="28da67938bcca5f6195c267cc5920aa5841f72a7e98769cf4d000b3966a164bb"
Sep 12 01:00:39.719106 ip-10-0-142-85 kubenswrapper[2558]: E0912 01:00:39.719063    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28da67938bcca5f6195c267cc5920aa5841f72a7e98769cf4d000b3966a164bb\": container with ID starting with 28da67938bcca5f6195c267cc5920aa5841f72a7e98769cf4d000b3966a164bb not found: ID does not exist" containerID="28da67938bcca5f6195c267cc5920aa5841f72a7e98769cf4d000b3966a164bb"
Sep 12 01:00:39.719209 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.719116    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28da67938bcca5f6195c267cc5920aa5841f72a7e98769cf4d000b3966a164bb"} err="failed to get container status \"28da67938bcca5f6195c267cc5920aa5841f72a7e98769cf4d000b3966a164bb\": rpc error: code = NotFound desc = could not find container \"28da67938bcca5f6195c267cc5920aa5841f72a7e98769cf4d000b3966a164bb\": container with ID starting with 28da67938bcca5f6195c267cc5920aa5841f72a7e98769cf4d000b3966a164bb not found: ID does not exist"
Sep 12 01:00:39.719209 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.719139    2558 scope.go:117] "RemoveContainer" containerID="2840385bc6469773a23866db5c89163467724ab8dda017eeb021438e0b8dec61"
Sep 12 01:00:39.727678 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.727664    2558 scope.go:117] "RemoveContainer" containerID="2840385bc6469773a23866db5c89163467724ab8dda017eeb021438e0b8dec61"
Sep 12 01:00:39.727940 ip-10-0-142-85 kubenswrapper[2558]: E0912 01:00:39.727925    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2840385bc6469773a23866db5c89163467724ab8dda017eeb021438e0b8dec61\": container with ID starting with 2840385bc6469773a23866db5c89163467724ab8dda017eeb021438e0b8dec61 not found: ID does not exist" containerID="2840385bc6469773a23866db5c89163467724ab8dda017eeb021438e0b8dec61"
Sep 12 01:00:39.727984 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.727945    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2840385bc6469773a23866db5c89163467724ab8dda017eeb021438e0b8dec61"} err="failed to get container status \"2840385bc6469773a23866db5c89163467724ab8dda017eeb021438e0b8dec61\": rpc error: code = NotFound desc = could not find container \"2840385bc6469773a23866db5c89163467724ab8dda017eeb021438e0b8dec61\": container with ID starting with 2840385bc6469773a23866db5c89163467724ab8dda017eeb021438e0b8dec61 not found: ID does not exist"
Sep 12 01:00:39.727984 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.727959    2558 scope.go:117] "RemoveContainer" containerID="5d9a83395a585f6064824d61e28737904d865cc199ac9b03277b01816c7bd12b"
Sep 12 01:00:39.736835 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.736820    2558 scope.go:117] "RemoveContainer" containerID="5d9a83395a585f6064824d61e28737904d865cc199ac9b03277b01816c7bd12b"
Sep 12 01:00:39.737112 ip-10-0-142-85 kubenswrapper[2558]: E0912 01:00:39.737074    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d9a83395a585f6064824d61e28737904d865cc199ac9b03277b01816c7bd12b\": container with ID starting with 5d9a83395a585f6064824d61e28737904d865cc199ac9b03277b01816c7bd12b not found: ID does not exist" containerID="5d9a83395a585f6064824d61e28737904d865cc199ac9b03277b01816c7bd12b"
Sep 12 01:00:39.737188 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.737122    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d9a83395a585f6064824d61e28737904d865cc199ac9b03277b01816c7bd12b"} err="failed to get container status \"5d9a83395a585f6064824d61e28737904d865cc199ac9b03277b01816c7bd12b\": rpc error: code = NotFound desc = could not find container \"5d9a83395a585f6064824d61e28737904d865cc199ac9b03277b01816c7bd12b\": container with ID starting with 5d9a83395a585f6064824d61e28737904d865cc199ac9b03277b01816c7bd12b not found: ID does not exist"
Sep 12 01:00:39.739099 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.739064    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-876808-6494cfc7b9-xwsw7"]
Sep 12 01:00:39.739194 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.739126    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-876808-6494cfc7b9-xwsw7"]
Sep 12 01:00:39.748496 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.748476    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb"]
Sep 12 01:00:39.754439 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.754418    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-876808-66d8548fdc-jhfqb"]
Sep 12 01:00:39.767138 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.767115    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-876808-7c65579d9d-sqzzc"]
Sep 12 01:00:39.770212 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:39.770191    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-876808-7c65579d9d-sqzzc"]
Sep 12 01:00:40.386534 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:40.386440    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:41.095519 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:41.095483    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:41.737568 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:41.737540    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49a730e9-c6fe-4d17-bd64-233e1d3ba62a" path="/var/lib/kubelet/pods/49a730e9-c6fe-4d17-bd64-233e1d3ba62a/volumes"
Sep 12 01:00:41.737951 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:41.737920    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56fdc776-1e9f-4d9d-ac51-536cc781215e" path="/var/lib/kubelet/pods/56fdc776-1e9f-4d9d-ac51-536cc781215e/volumes"
Sep 12 01:00:41.738286 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:41.738271    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de5f8abf-8e88-43a0-9f50-e1785612895f" path="/var/lib/kubelet/pods/de5f8abf-8e88-43a0-9f50-e1785612895f/volumes"
Sep 12 01:00:41.876973 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:41.876939    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7679-b6d10c"]
Sep 12 01:00:41.877362 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:41.877349    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de5f8abf-8e88-43a0-9f50-e1785612895f" containerName="payload-processing"
Sep 12 01:00:41.877422 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:41.877363    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="de5f8abf-8e88-43a0-9f50-e1785612895f" containerName="payload-processing"
Sep 12 01:00:41.877422 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:41.877376    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7279318-9146-4ec3-8f17-b7994c21f31f" containerName="cleanup"
Sep 12 01:00:41.877422 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:41.877381    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7279318-9146-4ec3-8f17-b7994c21f31f" containerName="cleanup"
Sep 12 01:00:41.877422 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:41.877396    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56fdc776-1e9f-4d9d-ac51-536cc781215e" containerName="maas-api"
Sep 12 01:00:41.877422 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:41.877402    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="56fdc776-1e9f-4d9d-ac51-536cc781215e" containerName="maas-api"
Sep 12 01:00:41.877422 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:41.877412    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49a730e9-c6fe-4d17-bd64-233e1d3ba62a" containerName="payload-pre-processing"
Sep 12 01:00:41.877422 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:41.877417    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="49a730e9-c6fe-4d17-bd64-233e1d3ba62a" containerName="payload-pre-processing"
Sep 12 01:00:41.877661 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:41.877475    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="de5f8abf-8e88-43a0-9f50-e1785612895f" containerName="payload-processing"
Sep 12 01:00:41.877661 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:41.877482    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7279318-9146-4ec3-8f17-b7994c21f31f" containerName="cleanup"
Sep 12 01:00:41.877661 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:41.877489    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="49a730e9-c6fe-4d17-bd64-233e1d3ba62a" containerName="payload-pre-processing"
Sep 12 01:00:41.877661 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:41.877496    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="56fdc776-1e9f-4d9d-ac51-536cc781215e" containerName="maas-api"
Sep 12 01:00:41.882288 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:41.882270    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7679-b6d10c"
Sep 12 01:00:41.885594 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:41.885573    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tqwft\""
Sep 12 01:00:41.905339 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:41.905269    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7679-b6d10c"]
Sep 12 01:00:41.943938 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:41.943904    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zl2rb\" (UniqueName: \"kubernetes.io/projected/7da400b1-d519-4839-a35d-3c48a958f827-kube-api-access-zl2rb\") pod \"test-curl-7679-b6d10c\" (UID: \"7da400b1-d519-4839-a35d-3c48a958f827\") " pod="opendatahub/test-curl-7679-b6d10c"
Sep 12 01:00:42.044745 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:42.044659    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zl2rb\" (UniqueName: \"kubernetes.io/projected/7da400b1-d519-4839-a35d-3c48a958f827-kube-api-access-zl2rb\") pod \"test-curl-7679-b6d10c\" (UID: \"7da400b1-d519-4839-a35d-3c48a958f827\") " pod="opendatahub/test-curl-7679-b6d10c"
Sep 12 01:00:42.057046 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:42.057020    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zl2rb\" (UniqueName: \"kubernetes.io/projected/7da400b1-d519-4839-a35d-3c48a958f827-kube-api-access-zl2rb\") pod \"test-curl-7679-b6d10c\" (UID: \"7da400b1-d519-4839-a35d-3c48a958f827\") " pod="opendatahub/test-curl-7679-b6d10c"
Sep 12 01:00:42.191841 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:42.191805    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7679-b6d10c"
Sep 12 01:00:42.463411 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:42.463386    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7679-b6d10c"]
Sep 12 01:00:42.465693 ip-10-0-142-85 kubenswrapper[2558]: W0912 01:00:42.465667    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7da400b1_d519_4839_a35d_3c48a958f827.slice/crio-731da3ab1caad9ffb124f56f470539a9c796db8bab96f21b17ac515b14e41cdf WatchSource:0}: Error finding container 731da3ab1caad9ffb124f56f470539a9c796db8bab96f21b17ac515b14e41cdf: Status 404 returned error can't find the container with id 731da3ab1caad9ffb124f56f470539a9c796db8bab96f21b17ac515b14e41cdf
Sep 12 01:00:42.732032 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:42.731931    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7679-b6d10c" event={"ID":"7da400b1-d519-4839-a35d-3c48a958f827","Type":"ContainerStarted","Data":"84f0e71d86b90aae16da79856cbaf535795141575b55600dd6c76e104f21189e"}
Sep 12 01:00:42.732032 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:42.731977    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7679-b6d10c" event={"ID":"7da400b1-d519-4839-a35d-3c48a958f827","Type":"ContainerStarted","Data":"731da3ab1caad9ffb124f56f470539a9c796db8bab96f21b17ac515b14e41cdf"}
Sep 12 01:00:42.747292 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:42.747244    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7679-b6d10c" podStartSLOduration=1.7472263479999999 podStartE2EDuration="1.747226348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:00:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:00:42.745021374 +0000 UTC m=+1879.419080231" watchObservedRunningTime="2026-09-12 01:00:42.747226348 +0000 UTC m=+1879.421285266"
Sep 12 01:00:43.281242 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:43.281201    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk" podUID="4c294f5a-088e-4633-829d-ed229ff054e9" containerName="main" containerID="cri-o://93a7bebba33b96860a5c0270304aa6d6e9ff94d11fd58b9f0d53d864f7c6e68e" gracePeriod=45
Sep 12 01:00:43.309534 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:43.309504    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7679-b6d10c"]
Sep 12 01:00:43.313210 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:43.313182    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7679-b6d10c"]
Sep 12 01:00:43.541656 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:43.541623    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk"
Sep 12 01:00:43.557505 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:43.557189    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c294f5a-088e-4633-829d-ed229ff054e9-model-cache\") pod \"4c294f5a-088e-4633-829d-ed229ff054e9\" (UID: \"4c294f5a-088e-4633-829d-ed229ff054e9\") "
Sep 12 01:00:43.557505 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:43.557243    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c294f5a-088e-4633-829d-ed229ff054e9-home\") pod \"4c294f5a-088e-4633-829d-ed229ff054e9\" (UID: \"4c294f5a-088e-4633-829d-ed229ff054e9\") "
Sep 12 01:00:43.557505 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:43.557273    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c294f5a-088e-4633-829d-ed229ff054e9-tmp-dir\") pod \"4c294f5a-088e-4633-829d-ed229ff054e9\" (UID: \"4c294f5a-088e-4633-829d-ed229ff054e9\") "
Sep 12 01:00:43.557505 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:43.557324    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2bfq2\" (UniqueName: \"kubernetes.io/projected/4c294f5a-088e-4633-829d-ed229ff054e9-kube-api-access-2bfq2\") pod \"4c294f5a-088e-4633-829d-ed229ff054e9\" (UID: \"4c294f5a-088e-4633-829d-ed229ff054e9\") "
Sep 12 01:00:43.557505 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:43.557366    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c294f5a-088e-4633-829d-ed229ff054e9-dshm\") pod \"4c294f5a-088e-4633-829d-ed229ff054e9\" (UID: \"4c294f5a-088e-4633-829d-ed229ff054e9\") "
Sep 12 01:00:43.557505 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:43.557410    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c294f5a-088e-4633-829d-ed229ff054e9-tls-certs\") pod \"4c294f5a-088e-4633-829d-ed229ff054e9\" (UID: \"4c294f5a-088e-4633-829d-ed229ff054e9\") "
Sep 12 01:00:43.557943 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:43.557570    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c294f5a-088e-4633-829d-ed229ff054e9-model-cache" (OuterVolumeSpecName: "model-cache") pod "4c294f5a-088e-4633-829d-ed229ff054e9" (UID: "4c294f5a-088e-4633-829d-ed229ff054e9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:00:43.557943 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:43.557689    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c294f5a-088e-4633-829d-ed229ff054e9-home" (OuterVolumeSpecName: "home") pod "4c294f5a-088e-4633-829d-ed229ff054e9" (UID: "4c294f5a-088e-4633-829d-ed229ff054e9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:00:43.557943 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:43.557882    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c294f5a-088e-4633-829d-ed229ff054e9-model-cache\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:00:43.557943 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:43.557901    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c294f5a-088e-4633-829d-ed229ff054e9-home\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:00:43.558178 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:43.558110    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c294f5a-088e-4633-829d-ed229ff054e9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4c294f5a-088e-4633-829d-ed229ff054e9" (UID: "4c294f5a-088e-4633-829d-ed229ff054e9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:00:43.560138 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:43.560108    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c294f5a-088e-4633-829d-ed229ff054e9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4c294f5a-088e-4633-829d-ed229ff054e9" (UID: "4c294f5a-088e-4633-829d-ed229ff054e9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 01:00:43.560274 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:43.560242    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c294f5a-088e-4633-829d-ed229ff054e9-dshm" (OuterVolumeSpecName: "dshm") pod "4c294f5a-088e-4633-829d-ed229ff054e9" (UID: "4c294f5a-088e-4633-829d-ed229ff054e9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:00:43.560329 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:43.560281    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c294f5a-088e-4633-829d-ed229ff054e9-kube-api-access-2bfq2" (OuterVolumeSpecName: "kube-api-access-2bfq2") pod "4c294f5a-088e-4633-829d-ed229ff054e9" (UID: "4c294f5a-088e-4633-829d-ed229ff054e9"). InnerVolumeSpecName "kube-api-access-2bfq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:00:43.658575 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:43.658495    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c294f5a-088e-4633-829d-ed229ff054e9-tmp-dir\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:00:43.658575 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:43.658534    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bfq2\" (UniqueName: \"kubernetes.io/projected/4c294f5a-088e-4633-829d-ed229ff054e9-kube-api-access-2bfq2\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:00:43.658575 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:43.658551    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c294f5a-088e-4633-829d-ed229ff054e9-dshm\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:00:43.658575 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:43.658564    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c294f5a-088e-4633-829d-ed229ff054e9-tls-certs\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:00:43.739971 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:43.739932    2558 generic.go:356] "Generic (PLEG): container finished" podID="4c294f5a-088e-4633-829d-ed229ff054e9" containerID="93a7bebba33b96860a5c0270304aa6d6e9ff94d11fd58b9f0d53d864f7c6e68e" exitCode=0
Sep 12 01:00:43.740177 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:43.739978    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk"
Sep 12 01:00:43.740177 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:43.740021    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk" event={"ID":"4c294f5a-088e-4633-829d-ed229ff054e9","Type":"ContainerDied","Data":"93a7bebba33b96860a5c0270304aa6d6e9ff94d11fd58b9f0d53d864f7c6e68e"}
Sep 12 01:00:43.740177 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:43.740071    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk" event={"ID":"4c294f5a-088e-4633-829d-ed229ff054e9","Type":"ContainerDied","Data":"fc8288effc49c4c8c10f0edd07c375fd48bbdbc839a327f348a7745a499f01ff"}
Sep 12 01:00:43.740177 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:43.740113    2558 scope.go:117] "RemoveContainer" containerID="93a7bebba33b96860a5c0270304aa6d6e9ff94d11fd58b9f0d53d864f7c6e68e"
Sep 12 01:00:43.740417 ip-10-0-142-85 kubenswrapper[2558]: E0912 01:00:43.740270    2558 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7679-b6d10c" container="test-curl-7679-b6d10c"
Sep 12 01:00:43.750001 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:43.749981    2558 scope.go:117] "RemoveContainer" containerID="93a7bebba33b96860a5c0270304aa6d6e9ff94d11fd58b9f0d53d864f7c6e68e"
Sep 12 01:00:43.750347 ip-10-0-142-85 kubenswrapper[2558]: E0912 01:00:43.750278    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93a7bebba33b96860a5c0270304aa6d6e9ff94d11fd58b9f0d53d864f7c6e68e\": container with ID starting with 93a7bebba33b96860a5c0270304aa6d6e9ff94d11fd58b9f0d53d864f7c6e68e not found: ID does not exist" containerID="93a7bebba33b96860a5c0270304aa6d6e9ff94d11fd58b9f0d53d864f7c6e68e"
Sep 12 01:00:43.750347 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:43.750303    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93a7bebba33b96860a5c0270304aa6d6e9ff94d11fd58b9f0d53d864f7c6e68e"} err="failed to get container status \"93a7bebba33b96860a5c0270304aa6d6e9ff94d11fd58b9f0d53d864f7c6e68e\": rpc error: code = NotFound desc = could not find container \"93a7bebba33b96860a5c0270304aa6d6e9ff94d11fd58b9f0d53d864f7c6e68e\": container with ID starting with 93a7bebba33b96860a5c0270304aa6d6e9ff94d11fd58b9f0d53d864f7c6e68e not found: ID does not exist"
Sep 12 01:00:43.762913 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:43.762880    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk"]
Sep 12 01:00:43.766876 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:43.766853    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-876808/distinct-2-w4-876808-kserve-5c45757dff-d8jmk"]
Sep 12 01:00:44.746521 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:44.746485    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7679-b6d10c" podUID="7da400b1-d519-4839-a35d-3c48a958f827" containerName="test-curl-7679-b6d10c" containerID="cri-o://84f0e71d86b90aae16da79856cbaf535795141575b55600dd6c76e104f21189e" gracePeriod=2
Sep 12 01:00:44.996843 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:44.996761    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf"]
Sep 12 01:00:45.017560 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:45.017524    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:45.671892 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:45.671861    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:45.739788 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:45.739756    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c294f5a-088e-4633-829d-ed229ff054e9" path="/var/lib/kubelet/pods/4c294f5a-088e-4633-829d-ed229ff054e9/volumes"
Sep 12 01:00:45.740950 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:45.740927    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:46.008726 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:46.008689    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:46.925865 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:46.925842    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7679-b6d10c"
Sep 12 01:00:46.987039 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:46.987007    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd"]
Sep 12 01:00:46.988932 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:46.988901    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zl2rb\" (UniqueName: \"kubernetes.io/projected/7da400b1-d519-4839-a35d-3c48a958f827-kube-api-access-zl2rb\") pod \"7da400b1-d519-4839-a35d-3c48a958f827\" (UID: \"7da400b1-d519-4839-a35d-3c48a958f827\") "
Sep 12 01:00:46.991910 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:46.991881    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7da400b1-d519-4839-a35d-3c48a958f827-kube-api-access-zl2rb" (OuterVolumeSpecName: "kube-api-access-zl2rb") pod "7da400b1-d519-4839-a35d-3c48a958f827" (UID: "7da400b1-d519-4839-a35d-3c48a958f827"). InnerVolumeSpecName "kube-api-access-zl2rb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:00:47.015142 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:47.015044    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:00:47.090598 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:47.090561    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zl2rb\" (UniqueName: \"kubernetes.io/projected/7da400b1-d519-4839-a35d-3c48a958f827-kube-api-access-zl2rb\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:00:47.738176 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:47.738141    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7da400b1-d519-4839-a35d-3c48a958f827" path="/var/lib/kubelet/pods/7da400b1-d519-4839-a35d-3c48a958f827/volumes"
Sep 12 01:00:47.763213 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:47.763176    2558 generic.go:356] "Generic (PLEG): container finished" podID="7da400b1-d519-4839-a35d-3c48a958f827" containerID="84f0e71d86b90aae16da79856cbaf535795141575b55600dd6c76e104f21189e" exitCode=137
Sep 12 01:00:47.763213 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:47.763198    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7679-b6d10c"
Sep 12 01:00:47.763434 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:47.763245    2558 scope.go:117] "RemoveContainer" containerID="84f0e71d86b90aae16da79856cbaf535795141575b55600dd6c76e104f21189e"
Sep 12 01:00:47.771921 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:47.771900    2558 scope.go:117] "RemoveContainer" containerID="84f0e71d86b90aae16da79856cbaf535795141575b55600dd6c76e104f21189e"
Sep 12 01:00:47.772236 ip-10-0-142-85 kubenswrapper[2558]: E0912 01:00:47.772221    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84f0e71d86b90aae16da79856cbaf535795141575b55600dd6c76e104f21189e\": container with ID starting with 84f0e71d86b90aae16da79856cbaf535795141575b55600dd6c76e104f21189e not found: ID does not exist" containerID="84f0e71d86b90aae16da79856cbaf535795141575b55600dd6c76e104f21189e"
Sep 12 01:00:47.772293 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:47.772244    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84f0e71d86b90aae16da79856cbaf535795141575b55600dd6c76e104f21189e"} err="failed to get container status \"84f0e71d86b90aae16da79856cbaf535795141575b55600dd6c76e104f21189e\": rpc error: code = NotFound desc = could not find container \"84f0e71d86b90aae16da79856cbaf535795141575b55600dd6c76e104f21189e\": container with ID starting with 84f0e71d86b90aae16da79856cbaf535795141575b55600dd6c76e104f21189e not found: ID does not exist"
Sep 12 01:00:57.121337 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:00:57.121273    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:01:00.016590 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:00.016550    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf" podUID="62e1050f-0f91-41e1-af0c-5d3b3d73f515" containerName="main" containerID="cri-o://c8aef4659e957e2777466dbce246af54ea0501dd81aa7c25ff09cf2680ef1264" gracePeriod=45
Sep 12 01:01:00.029471 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:00.029430    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf" podUID="62e1050f-0f91-41e1-af0c-5d3b3d73f515" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": dial tcp 10.133.0.68:8000: connect: connection refused"
Sep 12 01:01:00.270508 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:00.270444    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf"
Sep 12 01:01:00.322555 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:00.322523    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62e1050f-0f91-41e1-af0c-5d3b3d73f515-dshm\") pod \"62e1050f-0f91-41e1-af0c-5d3b3d73f515\" (UID: \"62e1050f-0f91-41e1-af0c-5d3b3d73f515\") "
Sep 12 01:01:00.322739 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:00.322573    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62e1050f-0f91-41e1-af0c-5d3b3d73f515-model-cache\") pod \"62e1050f-0f91-41e1-af0c-5d3b3d73f515\" (UID: \"62e1050f-0f91-41e1-af0c-5d3b3d73f515\") "
Sep 12 01:01:00.322739 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:00.322627    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62e1050f-0f91-41e1-af0c-5d3b3d73f515-tmp-dir\") pod \"62e1050f-0f91-41e1-af0c-5d3b3d73f515\" (UID: \"62e1050f-0f91-41e1-af0c-5d3b3d73f515\") "
Sep 12 01:01:00.322739 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:00.322649    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62e1050f-0f91-41e1-af0c-5d3b3d73f515-tls-certs\") pod \"62e1050f-0f91-41e1-af0c-5d3b3d73f515\" (UID: \"62e1050f-0f91-41e1-af0c-5d3b3d73f515\") "
Sep 12 01:01:00.322739 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:00.322711    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dmtc9\" (UniqueName: \"kubernetes.io/projected/62e1050f-0f91-41e1-af0c-5d3b3d73f515-kube-api-access-dmtc9\") pod \"62e1050f-0f91-41e1-af0c-5d3b3d73f515\" (UID: \"62e1050f-0f91-41e1-af0c-5d3b3d73f515\") "
Sep 12 01:01:00.322962 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:00.322756    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62e1050f-0f91-41e1-af0c-5d3b3d73f515-home\") pod \"62e1050f-0f91-41e1-af0c-5d3b3d73f515\" (UID: \"62e1050f-0f91-41e1-af0c-5d3b3d73f515\") "
Sep 12 01:01:00.322962 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:00.322881    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62e1050f-0f91-41e1-af0c-5d3b3d73f515-model-cache" (OuterVolumeSpecName: "model-cache") pod "62e1050f-0f91-41e1-af0c-5d3b3d73f515" (UID: "62e1050f-0f91-41e1-af0c-5d3b3d73f515"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:01:00.322962 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:00.322894    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62e1050f-0f91-41e1-af0c-5d3b3d73f515-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "62e1050f-0f91-41e1-af0c-5d3b3d73f515" (UID: "62e1050f-0f91-41e1-af0c-5d3b3d73f515"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:01:00.323163 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:00.323106    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62e1050f-0f91-41e1-af0c-5d3b3d73f515-home" (OuterVolumeSpecName: "home") pod "62e1050f-0f91-41e1-af0c-5d3b3d73f515" (UID: "62e1050f-0f91-41e1-af0c-5d3b3d73f515"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:01:00.323163 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:00.323139    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62e1050f-0f91-41e1-af0c-5d3b3d73f515-model-cache\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:01:00.323163 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:00.323156    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62e1050f-0f91-41e1-af0c-5d3b3d73f515-tmp-dir\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:01:00.325025 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:00.324999    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62e1050f-0f91-41e1-af0c-5d3b3d73f515-dshm" (OuterVolumeSpecName: "dshm") pod "62e1050f-0f91-41e1-af0c-5d3b3d73f515" (UID: "62e1050f-0f91-41e1-af0c-5d3b3d73f515"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:01:00.325152 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:00.325042    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62e1050f-0f91-41e1-af0c-5d3b3d73f515-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "62e1050f-0f91-41e1-af0c-5d3b3d73f515" (UID: "62e1050f-0f91-41e1-af0c-5d3b3d73f515"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 01:01:00.325195 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:00.325153    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62e1050f-0f91-41e1-af0c-5d3b3d73f515-kube-api-access-dmtc9" (OuterVolumeSpecName: "kube-api-access-dmtc9") pod "62e1050f-0f91-41e1-af0c-5d3b3d73f515" (UID: "62e1050f-0f91-41e1-af0c-5d3b3d73f515"). InnerVolumeSpecName "kube-api-access-dmtc9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:01:00.423989 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:00.423933    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62e1050f-0f91-41e1-af0c-5d3b3d73f515-tls-certs\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:01:00.423989 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:00.423982    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmtc9\" (UniqueName: \"kubernetes.io/projected/62e1050f-0f91-41e1-af0c-5d3b3d73f515-kube-api-access-dmtc9\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:01:00.423989 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:00.423996    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62e1050f-0f91-41e1-af0c-5d3b3d73f515-home\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:01:00.424269 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:00.424007    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62e1050f-0f91-41e1-af0c-5d3b3d73f515-dshm\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:01:00.830551 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:00.830516    2558 generic.go:356] "Generic (PLEG): container finished" podID="62e1050f-0f91-41e1-af0c-5d3b3d73f515" containerID="c8aef4659e957e2777466dbce246af54ea0501dd81aa7c25ff09cf2680ef1264" exitCode=0
Sep 12 01:01:00.830738 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:00.830560    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf"
Sep 12 01:01:00.830738 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:00.830589    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf" event={"ID":"62e1050f-0f91-41e1-af0c-5d3b3d73f515","Type":"ContainerDied","Data":"c8aef4659e957e2777466dbce246af54ea0501dd81aa7c25ff09cf2680ef1264"}
Sep 12 01:01:00.830738 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:00.830622    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf" event={"ID":"62e1050f-0f91-41e1-af0c-5d3b3d73f515","Type":"ContainerDied","Data":"14cb6454c5a50d1fd0e6a099cc8c3691df27021a5405ed393b04c304715782ec"}
Sep 12 01:01:00.830738 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:00.830639    2558 scope.go:117] "RemoveContainer" containerID="c8aef4659e957e2777466dbce246af54ea0501dd81aa7c25ff09cf2680ef1264"
Sep 12 01:01:00.840319 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:00.840298    2558 scope.go:117] "RemoveContainer" containerID="c8aef4659e957e2777466dbce246af54ea0501dd81aa7c25ff09cf2680ef1264"
Sep 12 01:01:00.840603 ip-10-0-142-85 kubenswrapper[2558]: E0912 01:01:00.840578    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8aef4659e957e2777466dbce246af54ea0501dd81aa7c25ff09cf2680ef1264\": container with ID starting with c8aef4659e957e2777466dbce246af54ea0501dd81aa7c25ff09cf2680ef1264 not found: ID does not exist" containerID="c8aef4659e957e2777466dbce246af54ea0501dd81aa7c25ff09cf2680ef1264"
Sep 12 01:01:00.840653 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:00.840616    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8aef4659e957e2777466dbce246af54ea0501dd81aa7c25ff09cf2680ef1264"} err="failed to get container status \"c8aef4659e957e2777466dbce246af54ea0501dd81aa7c25ff09cf2680ef1264\": rpc error: code = NotFound desc = could not find container \"c8aef4659e957e2777466dbce246af54ea0501dd81aa7c25ff09cf2680ef1264\": container with ID starting with c8aef4659e957e2777466dbce246af54ea0501dd81aa7c25ff09cf2680ef1264 not found: ID does not exist"
Sep 12 01:01:00.850873 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:00.850846    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf"]
Sep 12 01:01:00.853005 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:00.852985    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-9511bd/auth-test-model-9511bd-kserve-6ff9d86bb7-7w4hf"]
Sep 12 01:01:01.738070 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:01.738039    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62e1050f-0f91-41e1-af0c-5d3b3d73f515" path="/var/lib/kubelet/pods/62e1050f-0f91-41e1-af0c-5d3b3d73f515/volumes"
Sep 12 01:01:02.008997 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:02.008873    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd" podUID="910b9e2d-fc1c-49d5-a191-064f56754798" containerName="main" containerID="cri-o://410b0911dbfd89469bac94e9bade963366112a583dae567b34d41c19ac96a393" gracePeriod=45
Sep 12 01:01:02.264553 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:02.264491    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd"
Sep 12 01:01:02.341389 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:02.341359    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-52z44\" (UniqueName: \"kubernetes.io/projected/910b9e2d-fc1c-49d5-a191-064f56754798-kube-api-access-52z44\") pod \"910b9e2d-fc1c-49d5-a191-064f56754798\" (UID: \"910b9e2d-fc1c-49d5-a191-064f56754798\") "
Sep 12 01:01:02.341568 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:02.341434    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/910b9e2d-fc1c-49d5-a191-064f56754798-tmp-dir\") pod \"910b9e2d-fc1c-49d5-a191-064f56754798\" (UID: \"910b9e2d-fc1c-49d5-a191-064f56754798\") "
Sep 12 01:01:02.341568 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:02.341491    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/910b9e2d-fc1c-49d5-a191-064f56754798-home\") pod \"910b9e2d-fc1c-49d5-a191-064f56754798\" (UID: \"910b9e2d-fc1c-49d5-a191-064f56754798\") "
Sep 12 01:01:02.341568 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:02.341522    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/910b9e2d-fc1c-49d5-a191-064f56754798-tls-certs\") pod \"910b9e2d-fc1c-49d5-a191-064f56754798\" (UID: \"910b9e2d-fc1c-49d5-a191-064f56754798\") "
Sep 12 01:01:02.341568 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:02.341548    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/910b9e2d-fc1c-49d5-a191-064f56754798-model-cache\") pod \"910b9e2d-fc1c-49d5-a191-064f56754798\" (UID: \"910b9e2d-fc1c-49d5-a191-064f56754798\") "
Sep 12 01:01:02.341724 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:02.341586    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/910b9e2d-fc1c-49d5-a191-064f56754798-dshm\") pod \"910b9e2d-fc1c-49d5-a191-064f56754798\" (UID: \"910b9e2d-fc1c-49d5-a191-064f56754798\") "
Sep 12 01:01:02.341724 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:02.341704    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/910b9e2d-fc1c-49d5-a191-064f56754798-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "910b9e2d-fc1c-49d5-a191-064f56754798" (UID: "910b9e2d-fc1c-49d5-a191-064f56754798"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:01:02.341833 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:02.341808    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/910b9e2d-fc1c-49d5-a191-064f56754798-home" (OuterVolumeSpecName: "home") pod "910b9e2d-fc1c-49d5-a191-064f56754798" (UID: "910b9e2d-fc1c-49d5-a191-064f56754798"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:01:02.341833 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:02.341821    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/910b9e2d-fc1c-49d5-a191-064f56754798-model-cache" (OuterVolumeSpecName: "model-cache") pod "910b9e2d-fc1c-49d5-a191-064f56754798" (UID: "910b9e2d-fc1c-49d5-a191-064f56754798"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:01:02.341941 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:02.341858    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/910b9e2d-fc1c-49d5-a191-064f56754798-tmp-dir\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:01:02.343840 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:02.343817    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/910b9e2d-fc1c-49d5-a191-064f56754798-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "910b9e2d-fc1c-49d5-a191-064f56754798" (UID: "910b9e2d-fc1c-49d5-a191-064f56754798"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 01:01:02.343944 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:02.343840    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/910b9e2d-fc1c-49d5-a191-064f56754798-kube-api-access-52z44" (OuterVolumeSpecName: "kube-api-access-52z44") pod "910b9e2d-fc1c-49d5-a191-064f56754798" (UID: "910b9e2d-fc1c-49d5-a191-064f56754798"). InnerVolumeSpecName "kube-api-access-52z44". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:01:02.343944 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:02.343888    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/910b9e2d-fc1c-49d5-a191-064f56754798-dshm" (OuterVolumeSpecName: "dshm") pod "910b9e2d-fc1c-49d5-a191-064f56754798" (UID: "910b9e2d-fc1c-49d5-a191-064f56754798"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:01:02.443103 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:02.443058    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/910b9e2d-fc1c-49d5-a191-064f56754798-tls-certs\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:01:02.443276 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:02.443119    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/910b9e2d-fc1c-49d5-a191-064f56754798-model-cache\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:01:02.443276 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:02.443128    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/910b9e2d-fc1c-49d5-a191-064f56754798-dshm\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:01:02.443276 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:02.443138    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52z44\" (UniqueName: \"kubernetes.io/projected/910b9e2d-fc1c-49d5-a191-064f56754798-kube-api-access-52z44\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:01:02.443276 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:02.443147    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/910b9e2d-fc1c-49d5-a191-064f56754798-home\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:01:02.844421 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:02.844384    2558 generic.go:356] "Generic (PLEG): container finished" podID="910b9e2d-fc1c-49d5-a191-064f56754798" containerID="410b0911dbfd89469bac94e9bade963366112a583dae567b34d41c19ac96a393" exitCode=0
Sep 12 01:01:02.844900 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:02.844432    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd"
Sep 12 01:01:02.844900 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:02.844476    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd" event={"ID":"910b9e2d-fc1c-49d5-a191-064f56754798","Type":"ContainerDied","Data":"410b0911dbfd89469bac94e9bade963366112a583dae567b34d41c19ac96a393"}
Sep 12 01:01:02.844900 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:02.844524    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd" event={"ID":"910b9e2d-fc1c-49d5-a191-064f56754798","Type":"ContainerDied","Data":"163b9d0e1427552c62cf12f9b0d5896260ee92856809f50ef22145a643e12b0d"}
Sep 12 01:01:02.844900 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:02.844548    2558 scope.go:117] "RemoveContainer" containerID="410b0911dbfd89469bac94e9bade963366112a583dae567b34d41c19ac96a393"
Sep 12 01:01:02.854274 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:02.854256    2558 scope.go:117] "RemoveContainer" containerID="410b0911dbfd89469bac94e9bade963366112a583dae567b34d41c19ac96a393"
Sep 12 01:01:02.854538 ip-10-0-142-85 kubenswrapper[2558]: E0912 01:01:02.854518    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"410b0911dbfd89469bac94e9bade963366112a583dae567b34d41c19ac96a393\": container with ID starting with 410b0911dbfd89469bac94e9bade963366112a583dae567b34d41c19ac96a393 not found: ID does not exist" containerID="410b0911dbfd89469bac94e9bade963366112a583dae567b34d41c19ac96a393"
Sep 12 01:01:02.854608 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:02.854548    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"410b0911dbfd89469bac94e9bade963366112a583dae567b34d41c19ac96a393"} err="failed to get container status \"410b0911dbfd89469bac94e9bade963366112a583dae567b34d41c19ac96a393\": rpc error: code = NotFound desc = could not find container \"410b0911dbfd89469bac94e9bade963366112a583dae567b34d41c19ac96a393\": container with ID starting with 410b0911dbfd89469bac94e9bade963366112a583dae567b34d41c19ac96a393 not found: ID does not exist"
Sep 12 01:01:02.866221 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:02.866195    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd"]
Sep 12 01:01:02.869635 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:02.869615    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-dd25dd/auth-test-model-dd25dd-kserve-7659474bff-mcxbd"]
Sep 12 01:01:03.737933 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:03.737897    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="910b9e2d-fc1c-49d5-a191-064f56754798" path="/var/lib/kubelet/pods/910b9e2d-fc1c-49d5-a191-064f56754798/volumes"
Sep 12 01:01:04.431804 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:04.431761    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:01:04.531307 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:04.531274    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:01:05.879166 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:05.879127    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:01:19.253410 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:19.253378    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:01:20.248184 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:20.248142    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:01:27.485094 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:27.485052    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:01:29.629870 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.629837    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts"]
Sep 12 01:01:29.630355 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.630254    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62e1050f-0f91-41e1-af0c-5d3b3d73f515" containerName="main"
Sep 12 01:01:29.630355 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.630266    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="62e1050f-0f91-41e1-af0c-5d3b3d73f515" containerName="main"
Sep 12 01:01:29.630355 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.630273    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="910b9e2d-fc1c-49d5-a191-064f56754798" containerName="main"
Sep 12 01:01:29.630355 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.630278    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="910b9e2d-fc1c-49d5-a191-064f56754798" containerName="main"
Sep 12 01:01:29.630355 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.630297    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c294f5a-088e-4633-829d-ed229ff054e9" containerName="main"
Sep 12 01:01:29.630355 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.630303    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c294f5a-088e-4633-829d-ed229ff054e9" containerName="main"
Sep 12 01:01:29.630355 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.630314    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7da400b1-d519-4839-a35d-3c48a958f827" containerName="test-curl-7679-b6d10c"
Sep 12 01:01:29.630355 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.630319    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7da400b1-d519-4839-a35d-3c48a958f827" containerName="test-curl-7679-b6d10c"
Sep 12 01:01:29.630777 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.630375    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c294f5a-088e-4633-829d-ed229ff054e9" containerName="main"
Sep 12 01:01:29.630777 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.630383    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7da400b1-d519-4839-a35d-3c48a958f827" containerName="test-curl-7679-b6d10c"
Sep 12 01:01:29.630777 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.630389    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="910b9e2d-fc1c-49d5-a191-064f56754798" containerName="main"
Sep 12 01:01:29.630777 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.630394    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="62e1050f-0f91-41e1-af0c-5d3b3d73f515" containerName="main"
Sep 12 01:01:29.633428 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.633409    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts"
Sep 12 01:01:29.635897 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.635873    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-dd25dd\"/\"kube-root-ca.crt\""
Sep 12 01:01:29.635897 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.635888    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-dd25dd\"/\"openshift-service-ca.crt\""
Sep 12 01:01:29.636989 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.636969    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-dd25dd\"/\"default-dockercfg-kjpwh\""
Sep 12 01:01:29.637114 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.637035    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-dd25dd\"/\"sub-test-model-dd25dd-kserve-self-signed-certs\""
Sep 12 01:01:29.642577 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.642554    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts"]
Sep 12 01:01:29.650851 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.650829    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:01:29.707075 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.707042    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9d1deff-a26e-4119-b992-5a5ba4345020-tmp-dir\") pod \"sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts\" (UID: \"b9d1deff-a26e-4119-b992-5a5ba4345020\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts"
Sep 12 01:01:29.707254 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.707135    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgsr7\" (UniqueName: \"kubernetes.io/projected/b9d1deff-a26e-4119-b992-5a5ba4345020-kube-api-access-zgsr7\") pod \"sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts\" (UID: \"b9d1deff-a26e-4119-b992-5a5ba4345020\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts"
Sep 12 01:01:29.707254 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.707199    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9d1deff-a26e-4119-b992-5a5ba4345020-home\") pod \"sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts\" (UID: \"b9d1deff-a26e-4119-b992-5a5ba4345020\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts"
Sep 12 01:01:29.707254 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.707227    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9d1deff-a26e-4119-b992-5a5ba4345020-model-cache\") pod \"sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts\" (UID: \"b9d1deff-a26e-4119-b992-5a5ba4345020\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts"
Sep 12 01:01:29.707417 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.707271    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9d1deff-a26e-4119-b992-5a5ba4345020-dshm\") pod \"sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts\" (UID: \"b9d1deff-a26e-4119-b992-5a5ba4345020\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts"
Sep 12 01:01:29.707417 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.707306    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9d1deff-a26e-4119-b992-5a5ba4345020-tls-certs\") pod \"sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts\" (UID: \"b9d1deff-a26e-4119-b992-5a5ba4345020\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts"
Sep 12 01:01:29.807999 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.807957    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgsr7\" (UniqueName: \"kubernetes.io/projected/b9d1deff-a26e-4119-b992-5a5ba4345020-kube-api-access-zgsr7\") pod \"sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts\" (UID: \"b9d1deff-a26e-4119-b992-5a5ba4345020\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts"
Sep 12 01:01:29.808228 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.808049    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9d1deff-a26e-4119-b992-5a5ba4345020-home\") pod \"sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts\" (UID: \"b9d1deff-a26e-4119-b992-5a5ba4345020\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts"
Sep 12 01:01:29.808228 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.808079    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9d1deff-a26e-4119-b992-5a5ba4345020-model-cache\") pod \"sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts\" (UID: \"b9d1deff-a26e-4119-b992-5a5ba4345020\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts"
Sep 12 01:01:29.808228 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.808159    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9d1deff-a26e-4119-b992-5a5ba4345020-dshm\") pod \"sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts\" (UID: \"b9d1deff-a26e-4119-b992-5a5ba4345020\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts"
Sep 12 01:01:29.808228 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.808192    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9d1deff-a26e-4119-b992-5a5ba4345020-tls-certs\") pod \"sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts\" (UID: \"b9d1deff-a26e-4119-b992-5a5ba4345020\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts"
Sep 12 01:01:29.808500 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.808258    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9d1deff-a26e-4119-b992-5a5ba4345020-tmp-dir\") pod \"sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts\" (UID: \"b9d1deff-a26e-4119-b992-5a5ba4345020\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts"
Sep 12 01:01:29.808562 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.808513    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9d1deff-a26e-4119-b992-5a5ba4345020-home\") pod \"sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts\" (UID: \"b9d1deff-a26e-4119-b992-5a5ba4345020\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts"
Sep 12 01:01:29.808562 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.808541    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9d1deff-a26e-4119-b992-5a5ba4345020-model-cache\") pod \"sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts\" (UID: \"b9d1deff-a26e-4119-b992-5a5ba4345020\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts"
Sep 12 01:01:29.808685 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.808672    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9d1deff-a26e-4119-b992-5a5ba4345020-tmp-dir\") pod \"sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts\" (UID: \"b9d1deff-a26e-4119-b992-5a5ba4345020\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts"
Sep 12 01:01:29.810744 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.810714    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9d1deff-a26e-4119-b992-5a5ba4345020-dshm\") pod \"sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts\" (UID: \"b9d1deff-a26e-4119-b992-5a5ba4345020\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts"
Sep 12 01:01:29.810905 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.810889    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9d1deff-a26e-4119-b992-5a5ba4345020-tls-certs\") pod \"sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts\" (UID: \"b9d1deff-a26e-4119-b992-5a5ba4345020\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts"
Sep 12 01:01:29.817002 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.816980    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgsr7\" (UniqueName: \"kubernetes.io/projected/b9d1deff-a26e-4119-b992-5a5ba4345020-kube-api-access-zgsr7\") pod \"sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts\" (UID: \"b9d1deff-a26e-4119-b992-5a5ba4345020\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts"
Sep 12 01:01:29.944445 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:29.944411    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts"
Sep 12 01:01:30.215561 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:30.215487    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts"]
Sep 12 01:01:30.217914 ip-10-0-142-85 kubenswrapper[2558]: W0912 01:01:30.217883    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb9d1deff_a26e_4119_b992_5a5ba4345020.slice/crio-1c62a4bca11f2a37a23e9172df589b1d59938a05c16c9fe31f88beff59c54ffe WatchSource:0}: Error finding container 1c62a4bca11f2a37a23e9172df589b1d59938a05c16c9fe31f88beff59c54ffe: Status 404 returned error can't find the container with id 1c62a4bca11f2a37a23e9172df589b1d59938a05c16c9fe31f88beff59c54ffe
Sep 12 01:01:30.996525 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:30.996488    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts" event={"ID":"b9d1deff-a26e-4119-b992-5a5ba4345020","Type":"ContainerStarted","Data":"a8c018fd2f86db083551a6ef57ae46b4c02f1087a41b80dfa2548e70041eed5d"}
Sep 12 01:01:30.996905 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:30.996531    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts" event={"ID":"b9d1deff-a26e-4119-b992-5a5ba4345020","Type":"ContainerStarted","Data":"1c62a4bca11f2a37a23e9172df589b1d59938a05c16c9fe31f88beff59c54ffe"}
Sep 12 01:01:31.016381 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:31.016321    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts" podStartSLOduration=2.01630501 podStartE2EDuration="2.01630501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:01:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:01:31.013698292 +0000 UTC m=+1927.687757176" watchObservedRunningTime="2026-09-12 01:01:31.01630501 +0000 UTC m=+1927.690363870"
Sep 12 01:01:38.145620 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:38.145571    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:01:38.246246 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:38.246212    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:01:39.944606 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:39.944562    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts"
Sep 12 01:01:39.945003 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:39.944620    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts"
Sep 12 01:01:39.957495 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:39.957471    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts"
Sep 12 01:01:40.051689 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:01:40.051653    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts"
Sep 12 01:02:10.247620 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:10.247586    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:02:11.390832 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:11.390799    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts"]
Sep 12 01:02:11.404804 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:11.404778    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:02:19.416813 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:19.416741    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:02:20.117353 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:20.117324    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:02:20.214231 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:20.214198    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:02:26.410314 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:26.410276    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts" podUID="b9d1deff-a26e-4119-b992-5a5ba4345020" containerName="main" containerID="cri-o://a8c018fd2f86db083551a6ef57ae46b4c02f1087a41b80dfa2548e70041eed5d" gracePeriod=45
Sep 12 01:02:26.672321 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:26.672246    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts"
Sep 12 01:02:26.752371 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:26.752336    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9d1deff-a26e-4119-b992-5a5ba4345020-model-cache\") pod \"b9d1deff-a26e-4119-b992-5a5ba4345020\" (UID: \"b9d1deff-a26e-4119-b992-5a5ba4345020\") "
Sep 12 01:02:26.752541 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:26.752376    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9d1deff-a26e-4119-b992-5a5ba4345020-tmp-dir\") pod \"b9d1deff-a26e-4119-b992-5a5ba4345020\" (UID: \"b9d1deff-a26e-4119-b992-5a5ba4345020\") "
Sep 12 01:02:26.752541 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:26.752431    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9d1deff-a26e-4119-b992-5a5ba4345020-dshm\") pod \"b9d1deff-a26e-4119-b992-5a5ba4345020\" (UID: \"b9d1deff-a26e-4119-b992-5a5ba4345020\") "
Sep 12 01:02:26.752541 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:26.752462    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zgsr7\" (UniqueName: \"kubernetes.io/projected/b9d1deff-a26e-4119-b992-5a5ba4345020-kube-api-access-zgsr7\") pod \"b9d1deff-a26e-4119-b992-5a5ba4345020\" (UID: \"b9d1deff-a26e-4119-b992-5a5ba4345020\") "
Sep 12 01:02:26.752541 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:26.752481    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9d1deff-a26e-4119-b992-5a5ba4345020-tls-certs\") pod \"b9d1deff-a26e-4119-b992-5a5ba4345020\" (UID: \"b9d1deff-a26e-4119-b992-5a5ba4345020\") "
Sep 12 01:02:26.752541 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:26.752498    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9d1deff-a26e-4119-b992-5a5ba4345020-home\") pod \"b9d1deff-a26e-4119-b992-5a5ba4345020\" (UID: \"b9d1deff-a26e-4119-b992-5a5ba4345020\") "
Sep 12 01:02:26.752825 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:26.752685    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9d1deff-a26e-4119-b992-5a5ba4345020-model-cache" (OuterVolumeSpecName: "model-cache") pod "b9d1deff-a26e-4119-b992-5a5ba4345020" (UID: "b9d1deff-a26e-4119-b992-5a5ba4345020"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:02:26.752825 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:26.752723    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9d1deff-a26e-4119-b992-5a5ba4345020-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b9d1deff-a26e-4119-b992-5a5ba4345020" (UID: "b9d1deff-a26e-4119-b992-5a5ba4345020"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:02:26.752825 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:26.752816    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9d1deff-a26e-4119-b992-5a5ba4345020-home" (OuterVolumeSpecName: "home") pod "b9d1deff-a26e-4119-b992-5a5ba4345020" (UID: "b9d1deff-a26e-4119-b992-5a5ba4345020"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:02:26.754668 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:26.754646    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9d1deff-a26e-4119-b992-5a5ba4345020-dshm" (OuterVolumeSpecName: "dshm") pod "b9d1deff-a26e-4119-b992-5a5ba4345020" (UID: "b9d1deff-a26e-4119-b992-5a5ba4345020"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:02:26.754761 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:26.754694    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9d1deff-a26e-4119-b992-5a5ba4345020-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b9d1deff-a26e-4119-b992-5a5ba4345020" (UID: "b9d1deff-a26e-4119-b992-5a5ba4345020"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 01:02:26.754859 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:26.754843    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9d1deff-a26e-4119-b992-5a5ba4345020-kube-api-access-zgsr7" (OuterVolumeSpecName: "kube-api-access-zgsr7") pod "b9d1deff-a26e-4119-b992-5a5ba4345020" (UID: "b9d1deff-a26e-4119-b992-5a5ba4345020"). InnerVolumeSpecName "kube-api-access-zgsr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:02:26.854074 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:26.854028    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9d1deff-a26e-4119-b992-5a5ba4345020-tmp-dir\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:02:26.854074 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:26.854066    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9d1deff-a26e-4119-b992-5a5ba4345020-dshm\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:02:26.854338 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:26.854080    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgsr7\" (UniqueName: \"kubernetes.io/projected/b9d1deff-a26e-4119-b992-5a5ba4345020-kube-api-access-zgsr7\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:02:26.854338 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:26.854126    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9d1deff-a26e-4119-b992-5a5ba4345020-tls-certs\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:02:26.854338 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:26.854139    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9d1deff-a26e-4119-b992-5a5ba4345020-home\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:02:26.854338 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:26.854220    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9d1deff-a26e-4119-b992-5a5ba4345020-model-cache\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:02:27.277068 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:27.277034    2558 generic.go:356] "Generic (PLEG): container finished" podID="b9d1deff-a26e-4119-b992-5a5ba4345020" containerID="a8c018fd2f86db083551a6ef57ae46b4c02f1087a41b80dfa2548e70041eed5d" exitCode=0
Sep 12 01:02:27.277245 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:27.277118    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts" event={"ID":"b9d1deff-a26e-4119-b992-5a5ba4345020","Type":"ContainerDied","Data":"a8c018fd2f86db083551a6ef57ae46b4c02f1087a41b80dfa2548e70041eed5d"}
Sep 12 01:02:27.277245 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:27.277169    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts" event={"ID":"b9d1deff-a26e-4119-b992-5a5ba4345020","Type":"ContainerDied","Data":"1c62a4bca11f2a37a23e9172df589b1d59938a05c16c9fe31f88beff59c54ffe"}
Sep 12 01:02:27.277245 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:27.277078    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts"
Sep 12 01:02:27.277245 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:27.277187    2558 scope.go:117] "RemoveContainer" containerID="a8c018fd2f86db083551a6ef57ae46b4c02f1087a41b80dfa2548e70041eed5d"
Sep 12 01:02:27.286123 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:27.286099    2558 scope.go:117] "RemoveContainer" containerID="a8c018fd2f86db083551a6ef57ae46b4c02f1087a41b80dfa2548e70041eed5d"
Sep 12 01:02:27.286404 ip-10-0-142-85 kubenswrapper[2558]: E0912 01:02:27.286385    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8c018fd2f86db083551a6ef57ae46b4c02f1087a41b80dfa2548e70041eed5d\": container with ID starting with a8c018fd2f86db083551a6ef57ae46b4c02f1087a41b80dfa2548e70041eed5d not found: ID does not exist" containerID="a8c018fd2f86db083551a6ef57ae46b4c02f1087a41b80dfa2548e70041eed5d"
Sep 12 01:02:27.286469 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:27.286413    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8c018fd2f86db083551a6ef57ae46b4c02f1087a41b80dfa2548e70041eed5d"} err="failed to get container status \"a8c018fd2f86db083551a6ef57ae46b4c02f1087a41b80dfa2548e70041eed5d\": rpc error: code = NotFound desc = could not find container \"a8c018fd2f86db083551a6ef57ae46b4c02f1087a41b80dfa2548e70041eed5d\": container with ID starting with a8c018fd2f86db083551a6ef57ae46b4c02f1087a41b80dfa2548e70041eed5d not found: ID does not exist"
Sep 12 01:02:27.312801 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:27.312767    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts"]
Sep 12 01:02:27.314890 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:27.314867    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-dd25dd/sub-test-model-dd25dd-kserve-56b6bcc9b9-bvbts"]
Sep 12 01:02:27.740415 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:27.740381    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9d1deff-a26e-4119-b992-5a5ba4345020" path="/var/lib/kubelet/pods/b9d1deff-a26e-4119-b992-5a5ba4345020/volumes"
Sep 12 01:02:41.118005 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:41.117970    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq"]
Sep 12 01:02:41.118487 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:41.118449    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9d1deff-a26e-4119-b992-5a5ba4345020" containerName="main"
Sep 12 01:02:41.118487 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:41.118464    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9d1deff-a26e-4119-b992-5a5ba4345020" containerName="main"
Sep 12 01:02:41.118560 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:41.118526    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9d1deff-a26e-4119-b992-5a5ba4345020" containerName="main"
Sep 12 01:02:41.123412 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:41.123394    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq"
Sep 12 01:02:41.127189 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:41.127164    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-dd25dd\"/\"rate-test-model-dd25dd-kserve-self-signed-certs\""
Sep 12 01:02:41.127189 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:41.127173    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-dd25dd\"/\"openshift-service-ca.crt\""
Sep 12 01:02:41.127415 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:41.127167    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-dd25dd\"/\"kube-root-ca.crt\""
Sep 12 01:02:41.127415 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:41.127170    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-dd25dd\"/\"default-dockercfg-kjpwh\""
Sep 12 01:02:41.131239 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:41.131218    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq"]
Sep 12 01:02:41.135964 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:41.135942    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:02:41.190163 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:41.190117    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpdw4\" (UniqueName: \"kubernetes.io/projected/121ef9ed-ad31-4807-8950-c73dd9864363-kube-api-access-vpdw4\") pod \"rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq\" (UID: \"121ef9ed-ad31-4807-8950-c73dd9864363\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq"
Sep 12 01:02:41.190421 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:41.190207    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/121ef9ed-ad31-4807-8950-c73dd9864363-home\") pod \"rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq\" (UID: \"121ef9ed-ad31-4807-8950-c73dd9864363\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq"
Sep 12 01:02:41.190421 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:41.190257    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/121ef9ed-ad31-4807-8950-c73dd9864363-model-cache\") pod \"rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq\" (UID: \"121ef9ed-ad31-4807-8950-c73dd9864363\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq"
Sep 12 01:02:41.190421 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:41.190285    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/121ef9ed-ad31-4807-8950-c73dd9864363-tls-certs\") pod \"rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq\" (UID: \"121ef9ed-ad31-4807-8950-c73dd9864363\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq"
Sep 12 01:02:41.190421 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:41.190307    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/121ef9ed-ad31-4807-8950-c73dd9864363-tmp-dir\") pod \"rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq\" (UID: \"121ef9ed-ad31-4807-8950-c73dd9864363\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq"
Sep 12 01:02:41.190421 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:41.190363    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/121ef9ed-ad31-4807-8950-c73dd9864363-dshm\") pod \"rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq\" (UID: \"121ef9ed-ad31-4807-8950-c73dd9864363\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq"
Sep 12 01:02:41.291502 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:41.291451    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/121ef9ed-ad31-4807-8950-c73dd9864363-tls-certs\") pod \"rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq\" (UID: \"121ef9ed-ad31-4807-8950-c73dd9864363\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq"
Sep 12 01:02:41.291502 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:41.291509    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/121ef9ed-ad31-4807-8950-c73dd9864363-tmp-dir\") pod \"rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq\" (UID: \"121ef9ed-ad31-4807-8950-c73dd9864363\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq"
Sep 12 01:02:41.291782 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:41.291544    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/121ef9ed-ad31-4807-8950-c73dd9864363-dshm\") pod \"rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq\" (UID: \"121ef9ed-ad31-4807-8950-c73dd9864363\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq"
Sep 12 01:02:41.291782 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:41.291598    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpdw4\" (UniqueName: \"kubernetes.io/projected/121ef9ed-ad31-4807-8950-c73dd9864363-kube-api-access-vpdw4\") pod \"rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq\" (UID: \"121ef9ed-ad31-4807-8950-c73dd9864363\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq"
Sep 12 01:02:41.291782 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:41.291628    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/121ef9ed-ad31-4807-8950-c73dd9864363-home\") pod \"rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq\" (UID: \"121ef9ed-ad31-4807-8950-c73dd9864363\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq"
Sep 12 01:02:41.291782 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:41.291662    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/121ef9ed-ad31-4807-8950-c73dd9864363-model-cache\") pod \"rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq\" (UID: \"121ef9ed-ad31-4807-8950-c73dd9864363\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq"
Sep 12 01:02:41.292055 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:41.292031    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/121ef9ed-ad31-4807-8950-c73dd9864363-tmp-dir\") pod \"rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq\" (UID: \"121ef9ed-ad31-4807-8950-c73dd9864363\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq"
Sep 12 01:02:41.292164 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:41.292042    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/121ef9ed-ad31-4807-8950-c73dd9864363-home\") pod \"rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq\" (UID: \"121ef9ed-ad31-4807-8950-c73dd9864363\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq"
Sep 12 01:02:41.292164 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:41.292129    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/121ef9ed-ad31-4807-8950-c73dd9864363-model-cache\") pod \"rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq\" (UID: \"121ef9ed-ad31-4807-8950-c73dd9864363\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq"
Sep 12 01:02:41.293986 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:41.293962    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/121ef9ed-ad31-4807-8950-c73dd9864363-dshm\") pod \"rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq\" (UID: \"121ef9ed-ad31-4807-8950-c73dd9864363\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq"
Sep 12 01:02:41.294224 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:41.294206    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/121ef9ed-ad31-4807-8950-c73dd9864363-tls-certs\") pod \"rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq\" (UID: \"121ef9ed-ad31-4807-8950-c73dd9864363\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq"
Sep 12 01:02:41.302474 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:41.302453    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpdw4\" (UniqueName: \"kubernetes.io/projected/121ef9ed-ad31-4807-8950-c73dd9864363-kube-api-access-vpdw4\") pod \"rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq\" (UID: \"121ef9ed-ad31-4807-8950-c73dd9864363\") " pod="ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq"
Sep 12 01:02:41.434600 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:41.434511    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq"
Sep 12 01:02:41.711109 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:41.711015    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq"]
Sep 12 01:02:41.713483 ip-10-0-142-85 kubenswrapper[2558]: W0912 01:02:41.713453    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod121ef9ed_ad31_4807_8950_c73dd9864363.slice/crio-1afe30de95f599ebec686cbae4d08b16d7416d6562c056714ac4a84a07869f1e WatchSource:0}: Error finding container 1afe30de95f599ebec686cbae4d08b16d7416d6562c056714ac4a84a07869f1e: Status 404 returned error can't find the container with id 1afe30de95f599ebec686cbae4d08b16d7416d6562c056714ac4a84a07869f1e
Sep 12 01:02:42.353316 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:42.353278    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq" event={"ID":"121ef9ed-ad31-4807-8950-c73dd9864363","Type":"ContainerStarted","Data":"58e281349503d832e853af1b4775fb7965a869602620e62e107bcbdaabe7d7d6"}
Sep 12 01:02:42.353316 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:42.353324    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq" event={"ID":"121ef9ed-ad31-4807-8950-c73dd9864363","Type":"ContainerStarted","Data":"1afe30de95f599ebec686cbae4d08b16d7416d6562c056714ac4a84a07869f1e"}
Sep 12 01:02:42.370783 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:42.370729    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq" podStartSLOduration=1.3707115970000001 podStartE2EDuration="1.370711597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:02:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:02:42.369337806 +0000 UTC m=+1999.043396669" watchObservedRunningTime="2026-09-12 01:02:42.370711597 +0000 UTC m=+1999.044770456"
Sep 12 01:02:51.435006 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:51.434966    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq"
Sep 12 01:02:51.435440 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:51.435066    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq"
Sep 12 01:02:51.447673 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:51.447646    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq"
Sep 12 01:02:52.415741 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:02:52.415714    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq"
Sep 12 01:03:14.763157 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:14.763116    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:14.864278 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:14.864243    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:15.632338 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:15.632304    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq"]
Sep 12 01:03:15.641386 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:15.641352    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:16.082791 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:16.082755    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:16.383207 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:16.383063    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:16.478976 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:16.478945    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:18.479544 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:18.479492    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:19.251485 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:19.251449    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:19.337244 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:19.337213    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:20.275804 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:20.275768    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:21.038592 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:21.038553    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:21.101412 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:21.101369    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:21.345598 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:21.345504    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:21.534219 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:21.534184    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:21.634692 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:21.634600    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:21.663014 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:21.662975    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-9a2657-88bfd9b9c-j657t"]
Sep 12 01:03:21.666436 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:21.666413    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-9a2657-88bfd9b9c-j657t"
Sep 12 01:03:21.669114 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:21.669095    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-9a2657-dockercfg-g5ghz\""
Sep 12 01:03:21.669474 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:21.669456    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-9a2657\""
Sep 12 01:03:21.679558 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:21.679520    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-9a2657-88bfd9b9c-j657t"]
Sep 12 01:03:21.774157 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:21.774115    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwhxp\" (UniqueName: \"kubernetes.io/projected/bd4d2e3c-500f-495a-8eb9-331daae91b15-kube-api-access-cwhxp\") pod \"payload-processing-e2e-ipp-a-9a2657-88bfd9b9c-j657t\" (UID: \"bd4d2e3c-500f-495a-8eb9-331daae91b15\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-9a2657-88bfd9b9c-j657t"
Sep 12 01:03:21.774347 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:21.774200    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd4d2e3c-500f-495a-8eb9-331daae91b15-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-9a2657-88bfd9b9c-j657t\" (UID: \"bd4d2e3c-500f-495a-8eb9-331daae91b15\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-9a2657-88bfd9b9c-j657t"
Sep 12 01:03:21.875246 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:21.875205    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwhxp\" (UniqueName: \"kubernetes.io/projected/bd4d2e3c-500f-495a-8eb9-331daae91b15-kube-api-access-cwhxp\") pod \"payload-processing-e2e-ipp-a-9a2657-88bfd9b9c-j657t\" (UID: \"bd4d2e3c-500f-495a-8eb9-331daae91b15\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-9a2657-88bfd9b9c-j657t"
Sep 12 01:03:21.875428 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:21.875261    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd4d2e3c-500f-495a-8eb9-331daae91b15-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-9a2657-88bfd9b9c-j657t\" (UID: \"bd4d2e3c-500f-495a-8eb9-331daae91b15\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-9a2657-88bfd9b9c-j657t"
Sep 12 01:03:21.875980 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:21.875961    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd4d2e3c-500f-495a-8eb9-331daae91b15-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-9a2657-88bfd9b9c-j657t\" (UID: \"bd4d2e3c-500f-495a-8eb9-331daae91b15\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-9a2657-88bfd9b9c-j657t"
Sep 12 01:03:21.885013 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:21.884936    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwhxp\" (UniqueName: \"kubernetes.io/projected/bd4d2e3c-500f-495a-8eb9-331daae91b15-kube-api-access-cwhxp\") pod \"payload-processing-e2e-ipp-a-9a2657-88bfd9b9c-j657t\" (UID: \"bd4d2e3c-500f-495a-8eb9-331daae91b15\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-9a2657-88bfd9b9c-j657t"
Sep 12 01:03:21.979011 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:21.978965    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-9a2657-88bfd9b9c-j657t"
Sep 12 01:03:22.252878 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:22.252852    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-9a2657-88bfd9b9c-j657t"]
Sep 12 01:03:22.254848 ip-10-0-142-85 kubenswrapper[2558]: W0912 01:03:22.254820    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd4d2e3c_500f_495a_8eb9_331daae91b15.slice/crio-eb9fafc5f208f20e3017985fd970c278bab6307c9beb0d9ae521451c3c17e9d0 WatchSource:0}: Error finding container eb9fafc5f208f20e3017985fd970c278bab6307c9beb0d9ae521451c3c17e9d0: Status 404 returned error can't find the container with id eb9fafc5f208f20e3017985fd970c278bab6307c9beb0d9ae521451c3c17e9d0
Sep 12 01:03:22.558805 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:22.558715    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-9a2657-88bfd9b9c-j657t" event={"ID":"bd4d2e3c-500f-495a-8eb9-331daae91b15","Type":"ContainerStarted","Data":"b93898bf8071f2a61a8bea8bae6fb2accf8bac990657232459f7be7e4c9f8443"}
Sep 12 01:03:22.558805 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:22.558778    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-9a2657-88bfd9b9c-j657t" event={"ID":"bd4d2e3c-500f-495a-8eb9-331daae91b15","Type":"ContainerStarted","Data":"eb9fafc5f208f20e3017985fd970c278bab6307c9beb0d9ae521451c3c17e9d0"}
Sep 12 01:03:22.559353 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:22.558824    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-9a2657-88bfd9b9c-j657t"
Sep 12 01:03:22.575649 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:22.575574    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-9a2657-88bfd9b9c-j657t" podStartSLOduration=1.575537213 podStartE2EDuration="1.575537213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:03:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:03:22.574568594 +0000 UTC m=+2039.248627456" watchObservedRunningTime="2026-09-12 01:03:22.575537213 +0000 UTC m=+2039.249596073"
Sep 12 01:03:30.652073 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:30.652020    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq" podUID="121ef9ed-ad31-4807-8950-c73dd9864363" containerName="main" containerID="cri-o://58e281349503d832e853af1b4775fb7965a869602620e62e107bcbdaabe7d7d6" gracePeriod=45
Sep 12 01:03:30.908968 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:30.908906    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq"
Sep 12 01:03:30.963291 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:30.963256    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/121ef9ed-ad31-4807-8950-c73dd9864363-model-cache\") pod \"121ef9ed-ad31-4807-8950-c73dd9864363\" (UID: \"121ef9ed-ad31-4807-8950-c73dd9864363\") "
Sep 12 01:03:30.963465 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:30.963318    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/121ef9ed-ad31-4807-8950-c73dd9864363-dshm\") pod \"121ef9ed-ad31-4807-8950-c73dd9864363\" (UID: \"121ef9ed-ad31-4807-8950-c73dd9864363\") "
Sep 12 01:03:30.963465 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:30.963346    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/121ef9ed-ad31-4807-8950-c73dd9864363-tmp-dir\") pod \"121ef9ed-ad31-4807-8950-c73dd9864363\" (UID: \"121ef9ed-ad31-4807-8950-c73dd9864363\") "
Sep 12 01:03:30.963465 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:30.963378    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/121ef9ed-ad31-4807-8950-c73dd9864363-tls-certs\") pod \"121ef9ed-ad31-4807-8950-c73dd9864363\" (UID: \"121ef9ed-ad31-4807-8950-c73dd9864363\") "
Sep 12 01:03:30.963465 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:30.963401    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/121ef9ed-ad31-4807-8950-c73dd9864363-home\") pod \"121ef9ed-ad31-4807-8950-c73dd9864363\" (UID: \"121ef9ed-ad31-4807-8950-c73dd9864363\") "
Sep 12 01:03:30.963694 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:30.963472    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vpdw4\" (UniqueName: \"kubernetes.io/projected/121ef9ed-ad31-4807-8950-c73dd9864363-kube-api-access-vpdw4\") pod \"121ef9ed-ad31-4807-8950-c73dd9864363\" (UID: \"121ef9ed-ad31-4807-8950-c73dd9864363\") "
Sep 12 01:03:30.963694 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:30.963572    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/121ef9ed-ad31-4807-8950-c73dd9864363-model-cache" (OuterVolumeSpecName: "model-cache") pod "121ef9ed-ad31-4807-8950-c73dd9864363" (UID: "121ef9ed-ad31-4807-8950-c73dd9864363"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:03:30.963694 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:30.963637    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/121ef9ed-ad31-4807-8950-c73dd9864363-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "121ef9ed-ad31-4807-8950-c73dd9864363" (UID: "121ef9ed-ad31-4807-8950-c73dd9864363"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:03:30.963853 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:30.963713    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/121ef9ed-ad31-4807-8950-c73dd9864363-home" (OuterVolumeSpecName: "home") pod "121ef9ed-ad31-4807-8950-c73dd9864363" (UID: "121ef9ed-ad31-4807-8950-c73dd9864363"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:03:30.963912 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:30.963858    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/121ef9ed-ad31-4807-8950-c73dd9864363-home\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:03:30.963912 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:30.963871    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/121ef9ed-ad31-4807-8950-c73dd9864363-model-cache\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:03:30.963912 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:30.963881    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/121ef9ed-ad31-4807-8950-c73dd9864363-tmp-dir\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:03:30.965650 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:30.965625    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/121ef9ed-ad31-4807-8950-c73dd9864363-dshm" (OuterVolumeSpecName: "dshm") pod "121ef9ed-ad31-4807-8950-c73dd9864363" (UID: "121ef9ed-ad31-4807-8950-c73dd9864363"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:03:30.965788 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:30.965672    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/121ef9ed-ad31-4807-8950-c73dd9864363-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "121ef9ed-ad31-4807-8950-c73dd9864363" (UID: "121ef9ed-ad31-4807-8950-c73dd9864363"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 01:03:30.965788 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:30.965703    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/121ef9ed-ad31-4807-8950-c73dd9864363-kube-api-access-vpdw4" (OuterVolumeSpecName: "kube-api-access-vpdw4") pod "121ef9ed-ad31-4807-8950-c73dd9864363" (UID: "121ef9ed-ad31-4807-8950-c73dd9864363"). InnerVolumeSpecName "kube-api-access-vpdw4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:03:31.065435 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:31.065396    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/121ef9ed-ad31-4807-8950-c73dd9864363-dshm\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:03:31.065435 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:31.065430    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/121ef9ed-ad31-4807-8950-c73dd9864363-tls-certs\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:03:31.065435 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:31.065442    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vpdw4\" (UniqueName: \"kubernetes.io/projected/121ef9ed-ad31-4807-8950-c73dd9864363-kube-api-access-vpdw4\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:03:31.602764 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:31.602730    2558 generic.go:356] "Generic (PLEG): container finished" podID="121ef9ed-ad31-4807-8950-c73dd9864363" containerID="58e281349503d832e853af1b4775fb7965a869602620e62e107bcbdaabe7d7d6" exitCode=0
Sep 12 01:03:31.602967 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:31.602769    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq"
Sep 12 01:03:31.602967 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:31.602822    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq" event={"ID":"121ef9ed-ad31-4807-8950-c73dd9864363","Type":"ContainerDied","Data":"58e281349503d832e853af1b4775fb7965a869602620e62e107bcbdaabe7d7d6"}
Sep 12 01:03:31.602967 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:31.602864    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq" event={"ID":"121ef9ed-ad31-4807-8950-c73dd9864363","Type":"ContainerDied","Data":"1afe30de95f599ebec686cbae4d08b16d7416d6562c056714ac4a84a07869f1e"}
Sep 12 01:03:31.602967 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:31.602880    2558 scope.go:117] "RemoveContainer" containerID="58e281349503d832e853af1b4775fb7965a869602620e62e107bcbdaabe7d7d6"
Sep 12 01:03:31.612294 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:31.612274    2558 scope.go:117] "RemoveContainer" containerID="58e281349503d832e853af1b4775fb7965a869602620e62e107bcbdaabe7d7d6"
Sep 12 01:03:31.612551 ip-10-0-142-85 kubenswrapper[2558]: E0912 01:03:31.612533    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58e281349503d832e853af1b4775fb7965a869602620e62e107bcbdaabe7d7d6\": container with ID starting with 58e281349503d832e853af1b4775fb7965a869602620e62e107bcbdaabe7d7d6 not found: ID does not exist" containerID="58e281349503d832e853af1b4775fb7965a869602620e62e107bcbdaabe7d7d6"
Sep 12 01:03:31.612597 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:31.612560    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58e281349503d832e853af1b4775fb7965a869602620e62e107bcbdaabe7d7d6"} err="failed to get container status \"58e281349503d832e853af1b4775fb7965a869602620e62e107bcbdaabe7d7d6\": rpc error: code = NotFound desc = could not find container \"58e281349503d832e853af1b4775fb7965a869602620e62e107bcbdaabe7d7d6\": container with ID starting with 58e281349503d832e853af1b4775fb7965a869602620e62e107bcbdaabe7d7d6 not found: ID does not exist"
Sep 12 01:03:31.624143 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:31.624117    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq"]
Sep 12 01:03:31.628157 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:31.628136    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-dd25dd/rate-test-model-dd25dd-kserve-6dd648bd5f-gf7kq"]
Sep 12 01:03:31.738562 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:31.738517    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="121ef9ed-ad31-4807-8950-c73dd9864363" path="/var/lib/kubelet/pods/121ef9ed-ad31-4807-8950-c73dd9864363/volumes"
Sep 12 01:03:33.566286 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:33.566250    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-9a2657-88bfd9b9c-j657t"
Sep 12 01:03:43.272263 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:43.272223    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:44.190326 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:44.190283    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:44.290794 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:44.290756    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:44.920670 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:44.920635    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:45.303883 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:45.303808    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:45.412179 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:45.412149    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:45.548519 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:45.548471    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:46.169431 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:46.169371    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:47.557192 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:47.557156    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:48.254153 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:48.254078    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:48.353689 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:48.353643    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:48.555117 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:48.554982    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:49.838830 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:49.838789    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:50.716482 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:50.716437    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:51.462535 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:51.462491    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx"]
Sep 12 01:03:51.463190 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:51.463170    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="121ef9ed-ad31-4807-8950-c73dd9864363" containerName="main"
Sep 12 01:03:51.463242 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:51.463196    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="121ef9ed-ad31-4807-8950-c73dd9864363" containerName="main"
Sep 12 01:03:51.463344 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:51.463332    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="121ef9ed-ad31-4807-8950-c73dd9864363" containerName="main"
Sep 12 01:03:51.468606 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:51.468585    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx"
Sep 12 01:03:51.471286 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:51.471265    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-5b711c-serving-cert\""
Sep 12 01:03:51.478604 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:51.478580    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx"]
Sep 12 01:03:51.503138 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:51.503103    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:51.563022 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:51.562986    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/493f6f55-beab-458e-937e-d42fdf070fb5-metrics-certs\") pod \"maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx\" (UID: \"493f6f55-beab-458e-937e-d42fdf070fb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx"
Sep 12 01:03:51.563236 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:51.563063    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lgcw\" (UniqueName: \"kubernetes.io/projected/493f6f55-beab-458e-937e-d42fdf070fb5-kube-api-access-6lgcw\") pod \"maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx\" (UID: \"493f6f55-beab-458e-937e-d42fdf070fb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx"
Sep 12 01:03:51.563236 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:51.563176    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/493f6f55-beab-458e-937e-d42fdf070fb5-maas-api-tls\") pod \"maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx\" (UID: \"493f6f55-beab-458e-937e-d42fdf070fb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx"
Sep 12 01:03:51.563236 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:51.563233    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/493f6f55-beab-458e-937e-d42fdf070fb5-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx\" (UID: \"493f6f55-beab-458e-937e-d42fdf070fb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx"
Sep 12 01:03:51.585502 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:51.585464    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:51.664051 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:51.664020    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lgcw\" (UniqueName: \"kubernetes.io/projected/493f6f55-beab-458e-937e-d42fdf070fb5-kube-api-access-6lgcw\") pod \"maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx\" (UID: \"493f6f55-beab-458e-937e-d42fdf070fb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx"
Sep 12 01:03:51.664257 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:51.664110    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/493f6f55-beab-458e-937e-d42fdf070fb5-maas-api-tls\") pod \"maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx\" (UID: \"493f6f55-beab-458e-937e-d42fdf070fb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx"
Sep 12 01:03:51.664257 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:51.664174    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/493f6f55-beab-458e-937e-d42fdf070fb5-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx\" (UID: \"493f6f55-beab-458e-937e-d42fdf070fb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx"
Sep 12 01:03:51.664257 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:51.664212    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/493f6f55-beab-458e-937e-d42fdf070fb5-metrics-certs\") pod \"maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx\" (UID: \"493f6f55-beab-458e-937e-d42fdf070fb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx"
Sep 12 01:03:51.664848 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:51.664812    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/493f6f55-beab-458e-937e-d42fdf070fb5-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx\" (UID: \"493f6f55-beab-458e-937e-d42fdf070fb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx"
Sep 12 01:03:51.666688 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:51.666658    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/493f6f55-beab-458e-937e-d42fdf070fb5-maas-api-tls\") pod \"maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx\" (UID: \"493f6f55-beab-458e-937e-d42fdf070fb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx"
Sep 12 01:03:51.666795 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:51.666778    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/493f6f55-beab-458e-937e-d42fdf070fb5-metrics-certs\") pod \"maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx\" (UID: \"493f6f55-beab-458e-937e-d42fdf070fb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx"
Sep 12 01:03:51.673215 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:51.673196    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lgcw\" (UniqueName: \"kubernetes.io/projected/493f6f55-beab-458e-937e-d42fdf070fb5-kube-api-access-6lgcw\") pod \"maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx\" (UID: \"493f6f55-beab-458e-937e-d42fdf070fb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx"
Sep 12 01:03:51.747203 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:51.747052    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:51.781359 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:51.781322    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx"
Sep 12 01:03:52.059721 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:52.059615    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx"]
Sep 12 01:03:52.062049 ip-10-0-142-85 kubenswrapper[2558]: W0912 01:03:52.062022    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod493f6f55_beab_458e_937e_d42fdf070fb5.slice/crio-f04e73bb7e73c456eb0d7a1481062711169e53d1caa43d65b97f41d0118e4151 WatchSource:0}: Error finding container f04e73bb7e73c456eb0d7a1481062711169e53d1caa43d65b97f41d0118e4151: Status 404 returned error can't find the container with id f04e73bb7e73c456eb0d7a1481062711169e53d1caa43d65b97f41d0118e4151
Sep 12 01:03:52.063398 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:52.063378    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 12 01:03:52.069426 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:52.069405    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:52.166333 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:52.166294    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:52.243043 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:52.243010    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5b711c-6f6c894c89-88fpr"]
Sep 12 01:03:52.248113 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:52.248065    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5b711c-6f6c894c89-88fpr"
Sep 12 01:03:52.251428 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:52.251404    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-5b711c\""
Sep 12 01:03:52.251710 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:52.251692    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-5b711c-dockercfg-lj46q\""
Sep 12 01:03:52.266962 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:52.266937    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5b711c-6f6c894c89-88fpr"]
Sep 12 01:03:52.372584 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:52.372488    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59zhp\" (UniqueName: \"kubernetes.io/projected/15c4992e-a5a3-4069-be73-47ccf19075de-kube-api-access-59zhp\") pod \"payload-processing-e2e-ipp-b-5b711c-6f6c894c89-88fpr\" (UID: \"15c4992e-a5a3-4069-be73-47ccf19075de\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5b711c-6f6c894c89-88fpr"
Sep 12 01:03:52.372584 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:52.372534    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15c4992e-a5a3-4069-be73-47ccf19075de-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-5b711c-6f6c894c89-88fpr\" (UID: \"15c4992e-a5a3-4069-be73-47ccf19075de\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5b711c-6f6c894c89-88fpr"
Sep 12 01:03:52.473942 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:52.473905    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59zhp\" (UniqueName: \"kubernetes.io/projected/15c4992e-a5a3-4069-be73-47ccf19075de-kube-api-access-59zhp\") pod \"payload-processing-e2e-ipp-b-5b711c-6f6c894c89-88fpr\" (UID: \"15c4992e-a5a3-4069-be73-47ccf19075de\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5b711c-6f6c894c89-88fpr"
Sep 12 01:03:52.473942 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:52.473948    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15c4992e-a5a3-4069-be73-47ccf19075de-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-5b711c-6f6c894c89-88fpr\" (UID: \"15c4992e-a5a3-4069-be73-47ccf19075de\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5b711c-6f6c894c89-88fpr"
Sep 12 01:03:52.474601 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:52.474583    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15c4992e-a5a3-4069-be73-47ccf19075de-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-5b711c-6f6c894c89-88fpr\" (UID: \"15c4992e-a5a3-4069-be73-47ccf19075de\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5b711c-6f6c894c89-88fpr"
Sep 12 01:03:52.485283 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:52.485260    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59zhp\" (UniqueName: \"kubernetes.io/projected/15c4992e-a5a3-4069-be73-47ccf19075de-kube-api-access-59zhp\") pod \"payload-processing-e2e-ipp-b-5b711c-6f6c894c89-88fpr\" (UID: \"15c4992e-a5a3-4069-be73-47ccf19075de\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5b711c-6f6c894c89-88fpr"
Sep 12 01:03:52.559767 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:52.559739    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5b711c-6f6c894c89-88fpr"
Sep 12 01:03:52.711441 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:52.711409    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx" event={"ID":"493f6f55-beab-458e-937e-d42fdf070fb5","Type":"ContainerStarted","Data":"3fd6ae78d9a65665c75fd3169687ed5abfe19814ee04a185ebe933aa4d3c878e"}
Sep 12 01:03:52.711665 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:52.711465    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx" event={"ID":"493f6f55-beab-458e-937e-d42fdf070fb5","Type":"ContainerStarted","Data":"f04e73bb7e73c456eb0d7a1481062711169e53d1caa43d65b97f41d0118e4151"}
Sep 12 01:03:52.711665 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:52.711550    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx"
Sep 12 01:03:52.730470 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:52.730404    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx" podStartSLOduration=1.297020603 podStartE2EDuration="1.730386469s" totalImagesPullingTime="433.365866ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:03:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:03:52.72995806 +0000 UTC m=+2069.404016919" watchObservedRunningTime="2026-09-12 01:03:52.730386469 +0000 UTC m=+2069.404445329"
Sep 12 01:03:52.839313 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:52.839278    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5b711c-6f6c894c89-88fpr"]
Sep 12 01:03:52.840954 ip-10-0-142-85 kubenswrapper[2558]: W0912 01:03:52.840918    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15c4992e_a5a3_4069_be73_47ccf19075de.slice/crio-a30ca1ac920a0015cfca9abdd48dd5dec19d3376ab04d00308563e629f3a1552 WatchSource:0}: Error finding container a30ca1ac920a0015cfca9abdd48dd5dec19d3376ab04d00308563e629f3a1552: Status 404 returned error can't find the container with id a30ca1ac920a0015cfca9abdd48dd5dec19d3376ab04d00308563e629f3a1552
Sep 12 01:03:53.719981 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:53.719939    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5b711c-6f6c894c89-88fpr" event={"ID":"15c4992e-a5a3-4069-be73-47ccf19075de","Type":"ContainerStarted","Data":"a9821314046ad9d27d1dea82754cbbc0b66748e5a585fbe6b4dae56755d39ea3"}
Sep 12 01:03:53.720470 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:53.720011    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5b711c-6f6c894c89-88fpr" event={"ID":"15c4992e-a5a3-4069-be73-47ccf19075de","Type":"ContainerStarted","Data":"a30ca1ac920a0015cfca9abdd48dd5dec19d3376ab04d00308563e629f3a1552"}
Sep 12 01:03:53.720470 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:53.720270    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-5b711c-6f6c894c89-88fpr"
Sep 12 01:03:58.725872 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:58.725837    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx"
Sep 12 01:03:58.746889 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:58.746832    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-5b711c-6f6c894c89-88fpr" podStartSLOduration=6.746816441 podStartE2EDuration="6.746816441s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:03:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:03:53.752128634 +0000 UTC m=+2070.426187493" watchObservedRunningTime="2026-09-12 01:03:58.746816441 +0000 UTC m=+2075.420875300"
Sep 12 01:03:59.415739 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:59.415704    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:03:59.515992 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:03:59.515956    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:04:04.727499 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:04:04.727468    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-5b711c-6f6c894c89-88fpr"
Sep 12 01:04:13.923987 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:04:13.923950    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:04:14.625286 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:04:14.625246    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:04:15.345709 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:04:15.345674    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:04:16.072862 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:04:16.072826    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:04:16.786543 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:04:16.786498    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:04:18.295829 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:04:18.295773    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:04:23.433146 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:04:23.433113    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-5b711c-6f6c894c89-88fpr_15c4992e-a5a3-4069-be73-47ccf19075de/payload-processing/0.log"
Sep 12 01:04:23.897270 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:04:23.897130    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-czf8h_dd90996a-5428-4ccf-a154-c0aa8def9789/console-operator/1.log"
Sep 12 01:04:23.906822 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:04:23.906797    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-czf8h_dd90996a-5428-4ccf-a154-c0aa8def9789/console-operator/1.log"
Sep 12 01:04:26.783564 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:04:26.783514    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:04:33.341945 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:04:33.341908    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:04:36.046940 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:04:36.046900    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:01.135026 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:01.134972    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:09.450514 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:09.450483    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-9a2657-88bfd9b9c-j657t_bd4d2e3c-500f-495a-8eb9-331daae91b15/payload-processing/0.log"
Sep 12 01:05:09.683927 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:09.683902    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-5b711c-6f6c894c89-88fpr_15c4992e-a5a3-4069-be73-47ccf19075de/payload-processing/0.log"
Sep 12 01:05:09.949324 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:09.949289    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:10.241390 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:10.241299    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:11.002259 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:11.002218    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:12.885702 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:12.885657    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:13.590357 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:13.590321    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:13.681581 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:13.681549    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:13.791820 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:13.791792    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:14.018886 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:14.018848    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:14.634823 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:14.634789    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:15.579946 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:15.579865    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:16.270867 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:16.270830    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:16.972606 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:16.972567    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:17.080809 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:17.080775    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:17.174145 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:17.174110    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:17.436813 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:17.436769    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:17.504334 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:17.504304    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:17.583337 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:17.583303    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-783804-5556ddd99f-q6p76"]
Sep 12 01:05:17.589565 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:17.589531    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-783804-5556ddd99f-q6p76"
Sep 12 01:05:17.593456 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:17.593427    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-783804\""
Sep 12 01:05:17.593617 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:17.593441    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-783804-dockercfg-6ktzc\""
Sep 12 01:05:17.595880 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:17.595852    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbcd4b65-61de-42b2-a72d-1de5e9443b76-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-783804-5556ddd99f-q6p76\" (UID: \"bbcd4b65-61de-42b2-a72d-1de5e9443b76\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-783804-5556ddd99f-q6p76"
Sep 12 01:05:17.596224 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:17.596197    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6277\" (UniqueName: \"kubernetes.io/projected/bbcd4b65-61de-42b2-a72d-1de5e9443b76-kube-api-access-f6277\") pod \"payload-processing-e2e-ipp-cleanup-783804-5556ddd99f-q6p76\" (UID: \"bbcd4b65-61de-42b2-a72d-1de5e9443b76\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-783804-5556ddd99f-q6p76"
Sep 12 01:05:17.597381 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:17.597357    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-783804-5556ddd99f-q6p76"]
Sep 12 01:05:17.697552 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:17.697452    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbcd4b65-61de-42b2-a72d-1de5e9443b76-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-783804-5556ddd99f-q6p76\" (UID: \"bbcd4b65-61de-42b2-a72d-1de5e9443b76\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-783804-5556ddd99f-q6p76"
Sep 12 01:05:17.697740 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:17.697586    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6277\" (UniqueName: \"kubernetes.io/projected/bbcd4b65-61de-42b2-a72d-1de5e9443b76-kube-api-access-f6277\") pod \"payload-processing-e2e-ipp-cleanup-783804-5556ddd99f-q6p76\" (UID: \"bbcd4b65-61de-42b2-a72d-1de5e9443b76\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-783804-5556ddd99f-q6p76"
Sep 12 01:05:17.698235 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:17.698210    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbcd4b65-61de-42b2-a72d-1de5e9443b76-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-783804-5556ddd99f-q6p76\" (UID: \"bbcd4b65-61de-42b2-a72d-1de5e9443b76\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-783804-5556ddd99f-q6p76"
Sep 12 01:05:17.706072 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:17.706044    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6277\" (UniqueName: \"kubernetes.io/projected/bbcd4b65-61de-42b2-a72d-1de5e9443b76-kube-api-access-f6277\") pod \"payload-processing-e2e-ipp-cleanup-783804-5556ddd99f-q6p76\" (UID: \"bbcd4b65-61de-42b2-a72d-1de5e9443b76\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-783804-5556ddd99f-q6p76"
Sep 12 01:05:17.905002 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:17.904973    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-783804-5556ddd99f-q6p76"
Sep 12 01:05:18.185648 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:18.185614    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-783804-5556ddd99f-q6p76"]
Sep 12 01:05:18.187719 ip-10-0-142-85 kubenswrapper[2558]: W0912 01:05:18.187692    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbcd4b65_61de_42b2_a72d_1de5e9443b76.slice/crio-ff0f281a6e51aa960cbac4ff2f8e3611a19f4000ad8fe5bc157a40a91250a1f0 WatchSource:0}: Error finding container ff0f281a6e51aa960cbac4ff2f8e3611a19f4000ad8fe5bc157a40a91250a1f0: Status 404 returned error can't find the container with id ff0f281a6e51aa960cbac4ff2f8e3611a19f4000ad8fe5bc157a40a91250a1f0
Sep 12 01:05:19.158411 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:19.158371    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-783804-5556ddd99f-q6p76" event={"ID":"bbcd4b65-61de-42b2-a72d-1de5e9443b76","Type":"ContainerStarted","Data":"b66d69d5f3a5de83f463c5c09294c04d8acecf1fdb12d91ba2702743f563b281"}
Sep 12 01:05:19.158588 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:19.158434    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-783804-5556ddd99f-q6p76" event={"ID":"bbcd4b65-61de-42b2-a72d-1de5e9443b76","Type":"ContainerStarted","Data":"ff0f281a6e51aa960cbac4ff2f8e3611a19f4000ad8fe5bc157a40a91250a1f0"}
Sep 12 01:05:19.158588 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:19.158475    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-783804-5556ddd99f-q6p76"
Sep 12 01:05:19.180887 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:19.180836    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-783804-5556ddd99f-q6p76" podStartSLOduration=2.180820138 podStartE2EDuration="2.180820138s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:05:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:05:19.178026304 +0000 UTC m=+2155.852085163" watchObservedRunningTime="2026-09-12 01:05:19.180820138 +0000 UTC m=+2155.854878995"
Sep 12 01:05:30.165586 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:30.165554    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-783804-5556ddd99f-q6p76"
Sep 12 01:05:38.596167 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:38.596132    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:39.071103 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:39.071065    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:40.072596 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:40.072564    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:40.788275 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:40.788241    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:41.578226 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:41.578191    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:42.390860 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:42.390824    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:43.048466 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:43.048433    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:43.750697 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:43.750666    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:45.047318 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:45.047287    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:45.668935 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:45.668905    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:45.695481 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:45.695441    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-783804-5556ddd99f-q6p76"]
Sep 12 01:05:45.695809 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:45.695763    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-783804-5556ddd99f-q6p76" podUID="bbcd4b65-61de-42b2-a72d-1de5e9443b76" containerName="payload-processing" containerID="cri-o://b66d69d5f3a5de83f463c5c09294c04d8acecf1fdb12d91ba2702743f563b281" gracePeriod=30
Sep 12 01:05:45.769617 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:45.769587    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:45.870412 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:45.870364    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:45.943840 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:45.943807    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-783804-5556ddd99f-q6p76"
Sep 12 01:05:45.965037 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:45.965002    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:46.075298 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:46.075256    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbcd4b65-61de-42b2-a72d-1de5e9443b76-plugins-config-volume\") pod \"bbcd4b65-61de-42b2-a72d-1de5e9443b76\" (UID: \"bbcd4b65-61de-42b2-a72d-1de5e9443b76\") "
Sep 12 01:05:46.075298 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:46.075306    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6277\" (UniqueName: \"kubernetes.io/projected/bbcd4b65-61de-42b2-a72d-1de5e9443b76-kube-api-access-f6277\") pod \"bbcd4b65-61de-42b2-a72d-1de5e9443b76\" (UID: \"bbcd4b65-61de-42b2-a72d-1de5e9443b76\") "
Sep 12 01:05:46.075788 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:46.075643    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbcd4b65-61de-42b2-a72d-1de5e9443b76-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bbcd4b65-61de-42b2-a72d-1de5e9443b76" (UID: "bbcd4b65-61de-42b2-a72d-1de5e9443b76"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:05:46.077585 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:46.077558    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbcd4b65-61de-42b2-a72d-1de5e9443b76-kube-api-access-f6277" (OuterVolumeSpecName: "kube-api-access-f6277") pod "bbcd4b65-61de-42b2-a72d-1de5e9443b76" (UID: "bbcd4b65-61de-42b2-a72d-1de5e9443b76"). InnerVolumeSpecName "kube-api-access-f6277". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:05:46.176578 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:46.176479    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbcd4b65-61de-42b2-a72d-1de5e9443b76-plugins-config-volume\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:05:46.176578 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:46.176517    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6277\" (UniqueName: \"kubernetes.io/projected/bbcd4b65-61de-42b2-a72d-1de5e9443b76-kube-api-access-f6277\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:05:46.244346 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:46.244307    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:46.298983 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:46.298944    2558 generic.go:356] "Generic (PLEG): container finished" podID="bbcd4b65-61de-42b2-a72d-1de5e9443b76" containerID="b66d69d5f3a5de83f463c5c09294c04d8acecf1fdb12d91ba2702743f563b281" exitCode=0
Sep 12 01:05:46.299180 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:46.298992    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-783804-5556ddd99f-q6p76"
Sep 12 01:05:46.299180 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:46.299033    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-783804-5556ddd99f-q6p76" event={"ID":"bbcd4b65-61de-42b2-a72d-1de5e9443b76","Type":"ContainerDied","Data":"b66d69d5f3a5de83f463c5c09294c04d8acecf1fdb12d91ba2702743f563b281"}
Sep 12 01:05:46.299180 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:46.299102    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-783804-5556ddd99f-q6p76" event={"ID":"bbcd4b65-61de-42b2-a72d-1de5e9443b76","Type":"ContainerDied","Data":"ff0f281a6e51aa960cbac4ff2f8e3611a19f4000ad8fe5bc157a40a91250a1f0"}
Sep 12 01:05:46.299180 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:46.299126    2558 scope.go:117] "RemoveContainer" containerID="b66d69d5f3a5de83f463c5c09294c04d8acecf1fdb12d91ba2702743f563b281"
Sep 12 01:05:46.309125 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:46.309100    2558 scope.go:117] "RemoveContainer" containerID="b66d69d5f3a5de83f463c5c09294c04d8acecf1fdb12d91ba2702743f563b281"
Sep 12 01:05:46.309488 ip-10-0-142-85 kubenswrapper[2558]: E0912 01:05:46.309455    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b66d69d5f3a5de83f463c5c09294c04d8acecf1fdb12d91ba2702743f563b281\": container with ID starting with b66d69d5f3a5de83f463c5c09294c04d8acecf1fdb12d91ba2702743f563b281 not found: ID does not exist" containerID="b66d69d5f3a5de83f463c5c09294c04d8acecf1fdb12d91ba2702743f563b281"
Sep 12 01:05:46.309576 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:46.309497    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b66d69d5f3a5de83f463c5c09294c04d8acecf1fdb12d91ba2702743f563b281"} err="failed to get container status \"b66d69d5f3a5de83f463c5c09294c04d8acecf1fdb12d91ba2702743f563b281\": rpc error: code = NotFound desc = could not find container \"b66d69d5f3a5de83f463c5c09294c04d8acecf1fdb12d91ba2702743f563b281\": container with ID starting with b66d69d5f3a5de83f463c5c09294c04d8acecf1fdb12d91ba2702743f563b281 not found: ID does not exist"
Sep 12 01:05:46.320843 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:46.320812    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-783804-5556ddd99f-q6p76"]
Sep 12 01:05:46.325289 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:46.325266    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-783804-5556ddd99f-q6p76"]
Sep 12 01:05:46.379913 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:46.379881    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:47.158459 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:47.158423    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:47.737583 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:47.737549    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbcd4b65-61de-42b2-a72d-1de5e9443b76" path="/var/lib/kubelet/pods/bbcd4b65-61de-42b2-a72d-1de5e9443b76/volumes"
Sep 12 01:05:48.090966 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:48.090866    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:48.942643 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:48.942610    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:49.856526 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:49.856486    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:52.382878 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:52.382847    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:52.479865 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:52.479835    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:52.628127 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:52.628073    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:56.768685 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:56.768647    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:57.663271 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:57.663198    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:58.364484 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:58.364448    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:59.073020 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:59.072986    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:05:59.804935 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:05:59.804907    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:06:11.737304 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:06:11.737270    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:06:11.761346 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:06:11.761314    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-9a2657-88bfd9b9c-j657t"]
Sep 12 01:06:11.761646 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:06:11.761613    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-9a2657-88bfd9b9c-j657t" podUID="bd4d2e3c-500f-495a-8eb9-331daae91b15" containerName="payload-processing" containerID="cri-o://b93898bf8071f2a61a8bea8bae6fb2accf8bac990657232459f7be7e4c9f8443" gracePeriod=30
Sep 12 01:06:11.838349 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:06:11.838318    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:06:11.939324 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:06:11.939249    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:06:12.004773 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:06:12.004745    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:06:12.014707 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:06:12.014681    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-9a2657-88bfd9b9c-j657t"
Sep 12 01:06:12.121653 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:06:12.121622    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd4d2e3c-500f-495a-8eb9-331daae91b15-plugins-config-volume\") pod \"bd4d2e3c-500f-495a-8eb9-331daae91b15\" (UID: \"bd4d2e3c-500f-495a-8eb9-331daae91b15\") "
Sep 12 01:06:12.121839 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:06:12.121662    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cwhxp\" (UniqueName: \"kubernetes.io/projected/bd4d2e3c-500f-495a-8eb9-331daae91b15-kube-api-access-cwhxp\") pod \"bd4d2e3c-500f-495a-8eb9-331daae91b15\" (UID: \"bd4d2e3c-500f-495a-8eb9-331daae91b15\") "
Sep 12 01:06:12.122071 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:06:12.122044    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd4d2e3c-500f-495a-8eb9-331daae91b15-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bd4d2e3c-500f-495a-8eb9-331daae91b15" (UID: "bd4d2e3c-500f-495a-8eb9-331daae91b15"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:06:12.123914 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:06:12.123892    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd4d2e3c-500f-495a-8eb9-331daae91b15-kube-api-access-cwhxp" (OuterVolumeSpecName: "kube-api-access-cwhxp") pod "bd4d2e3c-500f-495a-8eb9-331daae91b15" (UID: "bd4d2e3c-500f-495a-8eb9-331daae91b15"). InnerVolumeSpecName "kube-api-access-cwhxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:06:12.222534 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:06:12.222494    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd4d2e3c-500f-495a-8eb9-331daae91b15-plugins-config-volume\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:06:12.222534 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:06:12.222528    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwhxp\" (UniqueName: \"kubernetes.io/projected/bd4d2e3c-500f-495a-8eb9-331daae91b15-kube-api-access-cwhxp\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:06:12.315407 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:06:12.315306    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:06:12.414981 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:06:12.414945    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:06:12.433682 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:06:12.433650    2558 generic.go:356] "Generic (PLEG): container finished" podID="bd4d2e3c-500f-495a-8eb9-331daae91b15" containerID="b93898bf8071f2a61a8bea8bae6fb2accf8bac990657232459f7be7e4c9f8443" exitCode=0
Sep 12 01:06:12.433855 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:06:12.433696    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-9a2657-88bfd9b9c-j657t"
Sep 12 01:06:12.433855 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:06:12.433702    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-9a2657-88bfd9b9c-j657t" event={"ID":"bd4d2e3c-500f-495a-8eb9-331daae91b15","Type":"ContainerDied","Data":"b93898bf8071f2a61a8bea8bae6fb2accf8bac990657232459f7be7e4c9f8443"}
Sep 12 01:06:12.433855 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:06:12.433753    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-9a2657-88bfd9b9c-j657t" event={"ID":"bd4d2e3c-500f-495a-8eb9-331daae91b15","Type":"ContainerDied","Data":"eb9fafc5f208f20e3017985fd970c278bab6307c9beb0d9ae521451c3c17e9d0"}
Sep 12 01:06:12.433855 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:06:12.433780    2558 scope.go:117] "RemoveContainer" containerID="b93898bf8071f2a61a8bea8bae6fb2accf8bac990657232459f7be7e4c9f8443"
Sep 12 01:06:12.443620 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:06:12.443603    2558 scope.go:117] "RemoveContainer" containerID="b93898bf8071f2a61a8bea8bae6fb2accf8bac990657232459f7be7e4c9f8443"
Sep 12 01:06:12.443893 ip-10-0-142-85 kubenswrapper[2558]: E0912 01:06:12.443877    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b93898bf8071f2a61a8bea8bae6fb2accf8bac990657232459f7be7e4c9f8443\": container with ID starting with b93898bf8071f2a61a8bea8bae6fb2accf8bac990657232459f7be7e4c9f8443 not found: ID does not exist" containerID="b93898bf8071f2a61a8bea8bae6fb2accf8bac990657232459f7be7e4c9f8443"
Sep 12 01:06:12.443943 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:06:12.443900    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b93898bf8071f2a61a8bea8bae6fb2accf8bac990657232459f7be7e4c9f8443"} err="failed to get container status \"b93898bf8071f2a61a8bea8bae6fb2accf8bac990657232459f7be7e4c9f8443\": rpc error: code = NotFound desc = could not find container \"b93898bf8071f2a61a8bea8bae6fb2accf8bac990657232459f7be7e4c9f8443\": container with ID starting with b93898bf8071f2a61a8bea8bae6fb2accf8bac990657232459f7be7e4c9f8443 not found: ID does not exist"
Sep 12 01:06:12.456364 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:06:12.456340    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-9a2657-88bfd9b9c-j657t"]
Sep 12 01:06:12.461017 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:06:12.460994    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-9a2657-88bfd9b9c-j657t"]
Sep 12 01:06:13.110399 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:06:13.110367    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:06:13.737560 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:06:13.737519    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd4d2e3c-500f-495a-8eb9-331daae91b15" path="/var/lib/kubelet/pods/bd4d2e3c-500f-495a-8eb9-331daae91b15/volumes"
Sep 12 01:06:13.908547 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:06:13.908512    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:06:14.613047 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:06:14.613008    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:06:17.349295 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:06:17.349250    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:06:23.114114 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:06:23.114062    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:06:53.840493 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:06:53.840398    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:06:54.780341 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:06:54.780311    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:00.336851 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:00.336813    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:00.395375 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:00.395330    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:00.632857 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:00.632755    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:01.282775 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:01.282737    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:01.982802 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:01.982771    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:02.675656 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:02.675620    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:03.459250 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:03.459209    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:04.128179 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:04.128142    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:11.486054 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:11.486016    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx"]
Sep 12 01:07:11.486552 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:11.486333    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx" podUID="493f6f55-beab-458e-937e-d42fdf070fb5" containerName="maas-api" containerID="cri-o://3fd6ae78d9a65665c75fd3169687ed5abfe19814ee04a185ebe933aa4d3c878e" gracePeriod=30
Sep 12 01:07:11.516973 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:11.516936    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:11.539191 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:11.539160    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5b711c-6f6c894c89-88fpr"]
Sep 12 01:07:11.539445 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:11.539416    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-5b711c-6f6c894c89-88fpr" podUID="15c4992e-a5a3-4069-be73-47ccf19075de" containerName="payload-processing" containerID="cri-o://a9821314046ad9d27d1dea82754cbbc0b66748e5a585fbe6b4dae56755d39ea3" gracePeriod=30
Sep 12 01:07:11.616846 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:11.616816    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:11.711988 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:11.711888    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:11.736295 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:11.736221    2558 generic.go:356] "Generic (PLEG): container finished" podID="493f6f55-beab-458e-937e-d42fdf070fb5" containerID="3fd6ae78d9a65665c75fd3169687ed5abfe19814ee04a185ebe933aa4d3c878e" exitCode=0
Sep 12 01:07:11.738853 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:11.738702    2558 generic.go:356] "Generic (PLEG): container finished" podID="15c4992e-a5a3-4069-be73-47ccf19075de" containerID="a9821314046ad9d27d1dea82754cbbc0b66748e5a585fbe6b4dae56755d39ea3" exitCode=0
Sep 12 01:07:11.740729 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:11.740696    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx" event={"ID":"493f6f55-beab-458e-937e-d42fdf070fb5","Type":"ContainerDied","Data":"3fd6ae78d9a65665c75fd3169687ed5abfe19814ee04a185ebe933aa4d3c878e"}
Sep 12 01:07:11.740869 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:11.740761    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5b711c-6f6c894c89-88fpr" event={"ID":"15c4992e-a5a3-4069-be73-47ccf19075de","Type":"ContainerDied","Data":"a9821314046ad9d27d1dea82754cbbc0b66748e5a585fbe6b4dae56755d39ea3"}
Sep 12 01:07:11.778781 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:11.778742    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:11.796990 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:11.796966    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx"
Sep 12 01:07:11.833197 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:11.833175    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5b711c-6f6c894c89-88fpr"
Sep 12 01:07:11.902633 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:11.902605    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6lgcw\" (UniqueName: \"kubernetes.io/projected/493f6f55-beab-458e-937e-d42fdf070fb5-kube-api-access-6lgcw\") pod \"493f6f55-beab-458e-937e-d42fdf070fb5\" (UID: \"493f6f55-beab-458e-937e-d42fdf070fb5\") "
Sep 12 01:07:11.902811 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:11.902650    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/493f6f55-beab-458e-937e-d42fdf070fb5-metrics-certs\") pod \"493f6f55-beab-458e-937e-d42fdf070fb5\" (UID: \"493f6f55-beab-458e-937e-d42fdf070fb5\") "
Sep 12 01:07:11.902811 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:11.902731    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/493f6f55-beab-458e-937e-d42fdf070fb5-maas-api-tls\") pod \"493f6f55-beab-458e-937e-d42fdf070fb5\" (UID: \"493f6f55-beab-458e-937e-d42fdf070fb5\") "
Sep 12 01:07:11.902947 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:11.902833    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/493f6f55-beab-458e-937e-d42fdf070fb5-openshift-service-ca\") pod \"493f6f55-beab-458e-937e-d42fdf070fb5\" (UID: \"493f6f55-beab-458e-937e-d42fdf070fb5\") "
Sep 12 01:07:11.904270 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:11.904234    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/493f6f55-beab-458e-937e-d42fdf070fb5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "493f6f55-beab-458e-937e-d42fdf070fb5" (UID: "493f6f55-beab-458e-937e-d42fdf070fb5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:07:11.905065 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:11.905041    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/493f6f55-beab-458e-937e-d42fdf070fb5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "493f6f55-beab-458e-937e-d42fdf070fb5" (UID: "493f6f55-beab-458e-937e-d42fdf070fb5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 01:07:11.905065 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:11.905049    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/493f6f55-beab-458e-937e-d42fdf070fb5-kube-api-access-6lgcw" (OuterVolumeSpecName: "kube-api-access-6lgcw") pod "493f6f55-beab-458e-937e-d42fdf070fb5" (UID: "493f6f55-beab-458e-937e-d42fdf070fb5"). InnerVolumeSpecName "kube-api-access-6lgcw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:07:11.905508 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:11.905490    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/493f6f55-beab-458e-937e-d42fdf070fb5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "493f6f55-beab-458e-937e-d42fdf070fb5" (UID: "493f6f55-beab-458e-937e-d42fdf070fb5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 01:07:12.004637 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:12.004543    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15c4992e-a5a3-4069-be73-47ccf19075de-plugins-config-volume\") pod \"15c4992e-a5a3-4069-be73-47ccf19075de\" (UID: \"15c4992e-a5a3-4069-be73-47ccf19075de\") "
Sep 12 01:07:12.004817 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:12.004669    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59zhp\" (UniqueName: \"kubernetes.io/projected/15c4992e-a5a3-4069-be73-47ccf19075de-kube-api-access-59zhp\") pod \"15c4992e-a5a3-4069-be73-47ccf19075de\" (UID: \"15c4992e-a5a3-4069-be73-47ccf19075de\") "
Sep 12 01:07:12.004901 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:12.004887    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/493f6f55-beab-458e-937e-d42fdf070fb5-maas-api-tls\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:07:12.004966 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:12.004903    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/493f6f55-beab-458e-937e-d42fdf070fb5-openshift-service-ca\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:07:12.004966 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:12.004913    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lgcw\" (UniqueName: \"kubernetes.io/projected/493f6f55-beab-458e-937e-d42fdf070fb5-kube-api-access-6lgcw\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:07:12.004966 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:12.004923    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/493f6f55-beab-458e-937e-d42fdf070fb5-metrics-certs\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:07:12.004966 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:12.004948    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15c4992e-a5a3-4069-be73-47ccf19075de-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "15c4992e-a5a3-4069-be73-47ccf19075de" (UID: "15c4992e-a5a3-4069-be73-47ccf19075de"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:07:12.006932 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:12.006905    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15c4992e-a5a3-4069-be73-47ccf19075de-kube-api-access-59zhp" (OuterVolumeSpecName: "kube-api-access-59zhp") pod "15c4992e-a5a3-4069-be73-47ccf19075de" (UID: "15c4992e-a5a3-4069-be73-47ccf19075de"). InnerVolumeSpecName "kube-api-access-59zhp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:07:12.106344 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:12.106293    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15c4992e-a5a3-4069-be73-47ccf19075de-plugins-config-volume\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:07:12.106344 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:12.106338    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59zhp\" (UniqueName: \"kubernetes.io/projected/15c4992e-a5a3-4069-be73-47ccf19075de-kube-api-access-59zhp\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:07:12.162790 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:12.162753    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:12.256020 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:12.255885    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:12.747074 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:12.747044    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5b711c-6f6c894c89-88fpr"
Sep 12 01:07:12.747579 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:12.747079    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5b711c-6f6c894c89-88fpr" event={"ID":"15c4992e-a5a3-4069-be73-47ccf19075de","Type":"ContainerDied","Data":"a30ca1ac920a0015cfca9abdd48dd5dec19d3376ab04d00308563e629f3a1552"}
Sep 12 01:07:12.747579 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:12.747165    2558 scope.go:117] "RemoveContainer" containerID="a9821314046ad9d27d1dea82754cbbc0b66748e5a585fbe6b4dae56755d39ea3"
Sep 12 01:07:12.749016 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:12.749000    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx"
Sep 12 01:07:12.749160 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:12.749002    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx" event={"ID":"493f6f55-beab-458e-937e-d42fdf070fb5","Type":"ContainerDied","Data":"f04e73bb7e73c456eb0d7a1481062711169e53d1caa43d65b97f41d0118e4151"}
Sep 12 01:07:12.757695 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:12.757679    2558 scope.go:117] "RemoveContainer" containerID="3fd6ae78d9a65665c75fd3169687ed5abfe19814ee04a185ebe933aa4d3c878e"
Sep 12 01:07:12.772878 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:12.772856    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx"]
Sep 12 01:07:12.777076 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:12.777054    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5b711c-78fff54865-knkjx"]
Sep 12 01:07:12.787870 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:12.787847    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5b711c-6f6c894c89-88fpr"]
Sep 12 01:07:12.795599 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:12.795576    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5b711c-6f6c894c89-88fpr"]
Sep 12 01:07:12.892875 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:12.892845    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:13.738957 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:13.738928    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15c4992e-a5a3-4069-be73-47ccf19075de" path="/var/lib/kubelet/pods/15c4992e-a5a3-4069-be73-47ccf19075de/volumes"
Sep 12 01:07:13.739358 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:13.739343    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="493f6f55-beab-458e-937e-d42fdf070fb5" path="/var/lib/kubelet/pods/493f6f55-beab-458e-937e-d42fdf070fb5/volumes"
Sep 12 01:07:13.751748 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:13.751713    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:18.998291 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:18.998261    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:19.122106 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:19.122042    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:19.361677 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:19.361590    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:19.982649 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:19.982611    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:21.063611 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:21.063577    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:22.015184 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:22.015152    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:23.336908 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:23.336876    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:30.246159 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.246121    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-9511bd-69d77c4d7f-6qgz6"]
Sep 12 01:07:30.246786 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.246380    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-9511bd-69d77c4d7f-6qgz6" podUID="d7311aa4-3f0a-45c1-a5ba-cf05195cfea8" containerName="payload-processing" containerID="cri-o://056de2fb8a17cb63ed3b3de44769dcf9182e36387611265bb3987c92430d9484" gracePeriod=30
Sep 12 01:07:30.253413 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.253381    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9511bd-8d458d974-85lsm"]
Sep 12 01:07:30.254717 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.254662    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9511bd-8d458d974-85lsm" podUID="d4eb4187-af72-4a2f-95de-3fd65ae9dd77" containerName="payload-pre-processing" containerID="cri-o://0bd73918029f9dba8030a8678e3000e7ff5f0bc2b69b814a9a7d527a436655d5" gracePeriod=30
Sep 12 01:07:30.270445 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.270383    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:30.370448 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.370416    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:30.428828 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.428790    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:30.541579 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.541542    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-9511bd-69d77c4d7f-6qgz6"
Sep 12 01:07:30.544497 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.544475    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9511bd-8d458d974-85lsm"
Sep 12 01:07:30.572166 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.572134    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djbv6\" (UniqueName: \"kubernetes.io/projected/d4eb4187-af72-4a2f-95de-3fd65ae9dd77-kube-api-access-djbv6\") pod \"d4eb4187-af72-4a2f-95de-3fd65ae9dd77\" (UID: \"d4eb4187-af72-4a2f-95de-3fd65ae9dd77\") "
Sep 12 01:07:30.572353 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.572273    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4eb4187-af72-4a2f-95de-3fd65ae9dd77-plugins-config-volume\") pod \"d4eb4187-af72-4a2f-95de-3fd65ae9dd77\" (UID: \"d4eb4187-af72-4a2f-95de-3fd65ae9dd77\") "
Sep 12 01:07:30.572353 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.572329    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7311aa4-3f0a-45c1-a5ba-cf05195cfea8-plugins-config-volume\") pod \"d7311aa4-3f0a-45c1-a5ba-cf05195cfea8\" (UID: \"d7311aa4-3f0a-45c1-a5ba-cf05195cfea8\") "
Sep 12 01:07:30.572473 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.572384    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5487g\" (UniqueName: \"kubernetes.io/projected/d7311aa4-3f0a-45c1-a5ba-cf05195cfea8-kube-api-access-5487g\") pod \"d7311aa4-3f0a-45c1-a5ba-cf05195cfea8\" (UID: \"d7311aa4-3f0a-45c1-a5ba-cf05195cfea8\") "
Sep 12 01:07:30.572714 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.572679    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4eb4187-af72-4a2f-95de-3fd65ae9dd77-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d4eb4187-af72-4a2f-95de-3fd65ae9dd77" (UID: "d4eb4187-af72-4a2f-95de-3fd65ae9dd77"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:07:30.572906 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.572763    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7311aa4-3f0a-45c1-a5ba-cf05195cfea8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d7311aa4-3f0a-45c1-a5ba-cf05195cfea8" (UID: "d7311aa4-3f0a-45c1-a5ba-cf05195cfea8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:07:30.575204 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.575170    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4eb4187-af72-4a2f-95de-3fd65ae9dd77-kube-api-access-djbv6" (OuterVolumeSpecName: "kube-api-access-djbv6") pod "d4eb4187-af72-4a2f-95de-3fd65ae9dd77" (UID: "d4eb4187-af72-4a2f-95de-3fd65ae9dd77"). InnerVolumeSpecName "kube-api-access-djbv6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:07:30.575561 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.575511    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7311aa4-3f0a-45c1-a5ba-cf05195cfea8-kube-api-access-5487g" (OuterVolumeSpecName: "kube-api-access-5487g") pod "d7311aa4-3f0a-45c1-a5ba-cf05195cfea8" (UID: "d7311aa4-3f0a-45c1-a5ba-cf05195cfea8"). InnerVolumeSpecName "kube-api-access-5487g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:07:30.674012 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.673977    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4eb4187-af72-4a2f-95de-3fd65ae9dd77-plugins-config-volume\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:07:30.674012 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.674007    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7311aa4-3f0a-45c1-a5ba-cf05195cfea8-plugins-config-volume\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:07:30.674012 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.674017    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5487g\" (UniqueName: \"kubernetes.io/projected/d7311aa4-3f0a-45c1-a5ba-cf05195cfea8-kube-api-access-5487g\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:07:30.674274 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.674030    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djbv6\" (UniqueName: \"kubernetes.io/projected/d4eb4187-af72-4a2f-95de-3fd65ae9dd77-kube-api-access-djbv6\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:07:30.784776 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.784670    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:30.850030 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.849998    2558 generic.go:356] "Generic (PLEG): container finished" podID="d4eb4187-af72-4a2f-95de-3fd65ae9dd77" containerID="0bd73918029f9dba8030a8678e3000e7ff5f0bc2b69b814a9a7d527a436655d5" exitCode=0
Sep 12 01:07:30.850256 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.850047    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9511bd-8d458d974-85lsm"
Sep 12 01:07:30.850256 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.850079    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9511bd-8d458d974-85lsm" event={"ID":"d4eb4187-af72-4a2f-95de-3fd65ae9dd77","Type":"ContainerDied","Data":"0bd73918029f9dba8030a8678e3000e7ff5f0bc2b69b814a9a7d527a436655d5"}
Sep 12 01:07:30.850256 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.850146    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9511bd-8d458d974-85lsm" event={"ID":"d4eb4187-af72-4a2f-95de-3fd65ae9dd77","Type":"ContainerDied","Data":"d8b2bd5d23ae36e0ed1ee4e1fe9291e7341b64ba74f81c3beab54a89c6b8c65b"}
Sep 12 01:07:30.850256 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.850177    2558 scope.go:117] "RemoveContainer" containerID="0bd73918029f9dba8030a8678e3000e7ff5f0bc2b69b814a9a7d527a436655d5"
Sep 12 01:07:30.851581 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.851558    2558 generic.go:356] "Generic (PLEG): container finished" podID="d7311aa4-3f0a-45c1-a5ba-cf05195cfea8" containerID="056de2fb8a17cb63ed3b3de44769dcf9182e36387611265bb3987c92430d9484" exitCode=0
Sep 12 01:07:30.851700 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.851596    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-9511bd-69d77c4d7f-6qgz6"
Sep 12 01:07:30.851700 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.851608    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-9511bd-69d77c4d7f-6qgz6" event={"ID":"d7311aa4-3f0a-45c1-a5ba-cf05195cfea8","Type":"ContainerDied","Data":"056de2fb8a17cb63ed3b3de44769dcf9182e36387611265bb3987c92430d9484"}
Sep 12 01:07:30.851700 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.851656    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-9511bd-69d77c4d7f-6qgz6" event={"ID":"d7311aa4-3f0a-45c1-a5ba-cf05195cfea8","Type":"ContainerDied","Data":"fcdf2a23110844404510a471fd8b77a02777cebe87d90044719b0a81f32cb242"}
Sep 12 01:07:30.860843 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.860817    2558 scope.go:117] "RemoveContainer" containerID="0bd73918029f9dba8030a8678e3000e7ff5f0bc2b69b814a9a7d527a436655d5"
Sep 12 01:07:30.861211 ip-10-0-142-85 kubenswrapper[2558]: E0912 01:07:30.861185    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bd73918029f9dba8030a8678e3000e7ff5f0bc2b69b814a9a7d527a436655d5\": container with ID starting with 0bd73918029f9dba8030a8678e3000e7ff5f0bc2b69b814a9a7d527a436655d5 not found: ID does not exist" containerID="0bd73918029f9dba8030a8678e3000e7ff5f0bc2b69b814a9a7d527a436655d5"
Sep 12 01:07:30.861311 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.861225    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bd73918029f9dba8030a8678e3000e7ff5f0bc2b69b814a9a7d527a436655d5"} err="failed to get container status \"0bd73918029f9dba8030a8678e3000e7ff5f0bc2b69b814a9a7d527a436655d5\": rpc error: code = NotFound desc = could not find container \"0bd73918029f9dba8030a8678e3000e7ff5f0bc2b69b814a9a7d527a436655d5\": container with ID starting with 0bd73918029f9dba8030a8678e3000e7ff5f0bc2b69b814a9a7d527a436655d5 not found: ID does not exist"
Sep 12 01:07:30.861311 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.861250    2558 scope.go:117] "RemoveContainer" containerID="056de2fb8a17cb63ed3b3de44769dcf9182e36387611265bb3987c92430d9484"
Sep 12 01:07:30.870204 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.870186    2558 scope.go:117] "RemoveContainer" containerID="056de2fb8a17cb63ed3b3de44769dcf9182e36387611265bb3987c92430d9484"
Sep 12 01:07:30.870482 ip-10-0-142-85 kubenswrapper[2558]: E0912 01:07:30.870465    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"056de2fb8a17cb63ed3b3de44769dcf9182e36387611265bb3987c92430d9484\": container with ID starting with 056de2fb8a17cb63ed3b3de44769dcf9182e36387611265bb3987c92430d9484 not found: ID does not exist" containerID="056de2fb8a17cb63ed3b3de44769dcf9182e36387611265bb3987c92430d9484"
Sep 12 01:07:30.870520 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.870503    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"056de2fb8a17cb63ed3b3de44769dcf9182e36387611265bb3987c92430d9484"} err="failed to get container status \"056de2fb8a17cb63ed3b3de44769dcf9182e36387611265bb3987c92430d9484\": rpc error: code = NotFound desc = could not find container \"056de2fb8a17cb63ed3b3de44769dcf9182e36387611265bb3987c92430d9484\": container with ID starting with 056de2fb8a17cb63ed3b3de44769dcf9182e36387611265bb3987c92430d9484 not found: ID does not exist"
Sep 12 01:07:30.877065 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.877042    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-9511bd-69d77c4d7f-6qgz6"]
Sep 12 01:07:30.882743 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.882721    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-9511bd-69d77c4d7f-6qgz6"]
Sep 12 01:07:30.884783 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.884760    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:30.892867 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.892845    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9511bd-8d458d974-85lsm"]
Sep 12 01:07:30.902632 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:30.902602    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-9511bd-8d458d974-85lsm"]
Sep 12 01:07:31.605973 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:31.605870    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:31.738302 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:31.738268    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4eb4187-af72-4a2f-95de-3fd65ae9dd77" path="/var/lib/kubelet/pods/d4eb4187-af72-4a2f-95de-3fd65ae9dd77/volumes"
Sep 12 01:07:31.738641 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:31.738628    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7311aa4-3f0a-45c1-a5ba-cf05195cfea8" path="/var/lib/kubelet/pods/d7311aa4-3f0a-45c1-a5ba-cf05195cfea8/volumes"
Sep 12 01:07:36.983692 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:36.983661    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:37.044358 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:37.044327    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:37.268911 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:37.268829    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:37.790739 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:37.790685    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:38.606530 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:38.606493    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:39.369579 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:39.369493    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:48.184944 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:48.184893    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:48.195601 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:48.195558    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-dd25dd-29819580-qxrx9"]
Sep 12 01:07:48.198747 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:48.198722    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-dd25dd-29819580-qxrx9"]
Sep 12 01:07:48.225504 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:48.225469    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-dd25dd-85477c9b8b-92knm"]
Sep 12 01:07:48.225791 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:48.225765    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-dd25dd-85477c9b8b-92knm" podUID="e2ef4545-d14b-4a5c-a2af-80962ed7b545" containerName="payload-pre-processing" containerID="cri-o://e18412f50547dbaf98df0474d689d49063ae169c8a479ab0c1f99ef110f87660" gracePeriod=30
Sep 12 01:07:48.285204 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:48.285154    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:48.385382 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:48.385346    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:48.445302 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:48.445205    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:48.483981 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:48.483956    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-dd25dd-85477c9b8b-92knm"
Sep 12 01:07:48.653754 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:48.653723    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cfg92\" (UniqueName: \"kubernetes.io/projected/e2ef4545-d14b-4a5c-a2af-80962ed7b545-kube-api-access-cfg92\") pod \"e2ef4545-d14b-4a5c-a2af-80962ed7b545\" (UID: \"e2ef4545-d14b-4a5c-a2af-80962ed7b545\") "
Sep 12 01:07:48.653949 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:48.653785    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2ef4545-d14b-4a5c-a2af-80962ed7b545-plugins-config-volume\") pod \"e2ef4545-d14b-4a5c-a2af-80962ed7b545\" (UID: \"e2ef4545-d14b-4a5c-a2af-80962ed7b545\") "
Sep 12 01:07:48.654230 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:48.654203    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2ef4545-d14b-4a5c-a2af-80962ed7b545-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e2ef4545-d14b-4a5c-a2af-80962ed7b545" (UID: "e2ef4545-d14b-4a5c-a2af-80962ed7b545"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:07:48.656093 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:48.656063    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2ef4545-d14b-4a5c-a2af-80962ed7b545-kube-api-access-cfg92" (OuterVolumeSpecName: "kube-api-access-cfg92") pod "e2ef4545-d14b-4a5c-a2af-80962ed7b545" (UID: "e2ef4545-d14b-4a5c-a2af-80962ed7b545"). InnerVolumeSpecName "kube-api-access-cfg92". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:07:48.755042 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:48.754949    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfg92\" (UniqueName: \"kubernetes.io/projected/e2ef4545-d14b-4a5c-a2af-80962ed7b545-kube-api-access-cfg92\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:07:48.755042 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:48.754984    2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2ef4545-d14b-4a5c-a2af-80962ed7b545-plugins-config-volume\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:07:48.792436 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:48.792383    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:48.895210 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:48.895173    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:48.952454 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:48.952417    2558 generic.go:356] "Generic (PLEG): container finished" podID="e2ef4545-d14b-4a5c-a2af-80962ed7b545" containerID="e18412f50547dbaf98df0474d689d49063ae169c8a479ab0c1f99ef110f87660" exitCode=0
Sep 12 01:07:48.952645 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:48.952464    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-dd25dd-85477c9b8b-92knm"
Sep 12 01:07:48.952645 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:48.952508    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-dd25dd-85477c9b8b-92knm" event={"ID":"e2ef4545-d14b-4a5c-a2af-80962ed7b545","Type":"ContainerDied","Data":"e18412f50547dbaf98df0474d689d49063ae169c8a479ab0c1f99ef110f87660"}
Sep 12 01:07:48.952645 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:48.952570    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-dd25dd-85477c9b8b-92knm" event={"ID":"e2ef4545-d14b-4a5c-a2af-80962ed7b545","Type":"ContainerDied","Data":"749ebcc0d739b76b08db51d927e6388c7d3db0ee38bd5df81110e7e52c082031"}
Sep 12 01:07:48.952645 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:48.952600    2558 scope.go:117] "RemoveContainer" containerID="e18412f50547dbaf98df0474d689d49063ae169c8a479ab0c1f99ef110f87660"
Sep 12 01:07:48.961607 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:48.961588    2558 scope.go:117] "RemoveContainer" containerID="e18412f50547dbaf98df0474d689d49063ae169c8a479ab0c1f99ef110f87660"
Sep 12 01:07:48.961897 ip-10-0-142-85 kubenswrapper[2558]: E0912 01:07:48.961877    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e18412f50547dbaf98df0474d689d49063ae169c8a479ab0c1f99ef110f87660\": container with ID starting with e18412f50547dbaf98df0474d689d49063ae169c8a479ab0c1f99ef110f87660 not found: ID does not exist" containerID="e18412f50547dbaf98df0474d689d49063ae169c8a479ab0c1f99ef110f87660"
Sep 12 01:07:48.961968 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:48.961910    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e18412f50547dbaf98df0474d689d49063ae169c8a479ab0c1f99ef110f87660"} err="failed to get container status \"e18412f50547dbaf98df0474d689d49063ae169c8a479ab0c1f99ef110f87660\": rpc error: code = NotFound desc = could not find container \"e18412f50547dbaf98df0474d689d49063ae169c8a479ab0c1f99ef110f87660\": container with ID starting with e18412f50547dbaf98df0474d689d49063ae169c8a479ab0c1f99ef110f87660 not found: ID does not exist"
Sep 12 01:07:48.976441 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:48.976388    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-dd25dd-85477c9b8b-92knm"]
Sep 12 01:07:48.983614 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:48.983586    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-dd25dd-85477c9b8b-92knm"]
Sep 12 01:07:49.737899 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:49.737865    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7279318-9146-4ec3-8f17-b7994c21f31f" path="/var/lib/kubelet/pods/a7279318-9146-4ec3-8f17-b7994c21f31f/volumes"
Sep 12 01:07:49.738302 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:49.738235    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2ef4545-d14b-4a5c-a2af-80962ed7b545" path="/var/lib/kubelet/pods/e2ef4545-d14b-4a5c-a2af-80962ed7b545/volumes"
Sep 12 01:07:54.709639 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:54.709600    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:54.812311 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:54.812259    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:54.983652 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:54.983569    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:07:57.699499 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:57.699468    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6cf675f9b4-t6jrs"]
Sep 12 01:07:57.699943 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:57.699681    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6cf675f9b4-t6jrs" podUID="9216d8e6-c84e-4651-91e6-41da9378e302" containerName="manager" containerID="cri-o://cf8773af148e0da6b496531270d792f55556031ae4a17094ef59e512720dde43" gracePeriod=30
Sep 12 01:07:57.953465 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:57.953401    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf675f9b4-t6jrs"
Sep 12 01:07:58.002780 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:58.002738    2558 generic.go:356] "Generic (PLEG): container finished" podID="9216d8e6-c84e-4651-91e6-41da9378e302" containerID="cf8773af148e0da6b496531270d792f55556031ae4a17094ef59e512720dde43" exitCode=0
Sep 12 01:07:58.002968 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:58.002825    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf675f9b4-t6jrs" event={"ID":"9216d8e6-c84e-4651-91e6-41da9378e302","Type":"ContainerDied","Data":"cf8773af148e0da6b496531270d792f55556031ae4a17094ef59e512720dde43"}
Sep 12 01:07:58.002968 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:58.002878    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf675f9b4-t6jrs" event={"ID":"9216d8e6-c84e-4651-91e6-41da9378e302","Type":"ContainerDied","Data":"fae4929eb8afa7187433c2a67ea5782a94dfe402afec2efb811a89b993cdd030"}
Sep 12 01:07:58.002968 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:58.002891    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf675f9b4-t6jrs"
Sep 12 01:07:58.003210 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:58.002904    2558 scope.go:117] "RemoveContainer" containerID="cf8773af148e0da6b496531270d792f55556031ae4a17094ef59e512720dde43"
Sep 12 01:07:58.013060 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:58.013039    2558 scope.go:117] "RemoveContainer" containerID="cf8773af148e0da6b496531270d792f55556031ae4a17094ef59e512720dde43"
Sep 12 01:07:58.013419 ip-10-0-142-85 kubenswrapper[2558]: E0912 01:07:58.013395    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf8773af148e0da6b496531270d792f55556031ae4a17094ef59e512720dde43\": container with ID starting with cf8773af148e0da6b496531270d792f55556031ae4a17094ef59e512720dde43 not found: ID does not exist" containerID="cf8773af148e0da6b496531270d792f55556031ae4a17094ef59e512720dde43"
Sep 12 01:07:58.013526 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:58.013426    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf8773af148e0da6b496531270d792f55556031ae4a17094ef59e512720dde43"} err="failed to get container status \"cf8773af148e0da6b496531270d792f55556031ae4a17094ef59e512720dde43\": rpc error: code = NotFound desc = could not find container \"cf8773af148e0da6b496531270d792f55556031ae4a17094ef59e512720dde43\": container with ID starting with cf8773af148e0da6b496531270d792f55556031ae4a17094ef59e512720dde43 not found: ID does not exist"
Sep 12 01:07:58.142287 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:58.142250    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9216d8e6-c84e-4651-91e6-41da9378e302-webhook-cert\") pod \"9216d8e6-c84e-4651-91e6-41da9378e302\" (UID: \"9216d8e6-c84e-4651-91e6-41da9378e302\") "
Sep 12 01:07:58.142469 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:58.142317    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9216d8e6-c84e-4651-91e6-41da9378e302-metrics-certs\") pod \"9216d8e6-c84e-4651-91e6-41da9378e302\" (UID: \"9216d8e6-c84e-4651-91e6-41da9378e302\") "
Sep 12 01:07:58.142469 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:58.142383    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7tnd8\" (UniqueName: \"kubernetes.io/projected/9216d8e6-c84e-4651-91e6-41da9378e302-kube-api-access-7tnd8\") pod \"9216d8e6-c84e-4651-91e6-41da9378e302\" (UID: \"9216d8e6-c84e-4651-91e6-41da9378e302\") "
Sep 12 01:07:58.144552 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:58.144517    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9216d8e6-c84e-4651-91e6-41da9378e302-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9216d8e6-c84e-4651-91e6-41da9378e302" (UID: "9216d8e6-c84e-4651-91e6-41da9378e302"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 01:07:58.144699 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:58.144643    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9216d8e6-c84e-4651-91e6-41da9378e302-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9216d8e6-c84e-4651-91e6-41da9378e302" (UID: "9216d8e6-c84e-4651-91e6-41da9378e302"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 01:07:58.144699 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:58.144660    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9216d8e6-c84e-4651-91e6-41da9378e302-kube-api-access-7tnd8" (OuterVolumeSpecName: "kube-api-access-7tnd8") pod "9216d8e6-c84e-4651-91e6-41da9378e302" (UID: "9216d8e6-c84e-4651-91e6-41da9378e302"). InnerVolumeSpecName "kube-api-access-7tnd8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:07:58.243640 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:58.243542    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tnd8\" (UniqueName: \"kubernetes.io/projected/9216d8e6-c84e-4651-91e6-41da9378e302-kube-api-access-7tnd8\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:07:58.243640 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:58.243584    2558 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9216d8e6-c84e-4651-91e6-41da9378e302-webhook-cert\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:07:58.243640 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:58.243593    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9216d8e6-c84e-4651-91e6-41da9378e302-metrics-certs\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:07:58.325121 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:58.325055    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6cf675f9b4-t6jrs"]
Sep 12 01:07:58.326882 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:58.326862    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6cf675f9b4-t6jrs"]
Sep 12 01:07:59.737936 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:07:59.737895    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9216d8e6-c84e-4651-91e6-41da9378e302" path="/var/lib/kubelet/pods/9216d8e6-c84e-4651-91e6-41da9378e302/volumes"
Sep 12 01:08:24.109980 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:24.109905    2558 scope.go:117] "RemoveContainer" containerID="1e782de04198789a5aa408a8664f292e8e075adb51a7ee19c77728a965ac0c1c"
Sep 12 01:08:59.180588 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.180558    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6cf675f9b4-kvqdh"]
Sep 12 01:08:59.181054 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.180940    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9216d8e6-c84e-4651-91e6-41da9378e302" containerName="manager"
Sep 12 01:08:59.181054 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.180953    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9216d8e6-c84e-4651-91e6-41da9378e302" containerName="manager"
Sep 12 01:08:59.181054 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.180965    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7311aa4-3f0a-45c1-a5ba-cf05195cfea8" containerName="payload-processing"
Sep 12 01:08:59.181054 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.180974    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7311aa4-3f0a-45c1-a5ba-cf05195cfea8" containerName="payload-processing"
Sep 12 01:08:59.181054 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.180985    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15c4992e-a5a3-4069-be73-47ccf19075de" containerName="payload-processing"
Sep 12 01:08:59.181054 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.180991    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="15c4992e-a5a3-4069-be73-47ccf19075de" containerName="payload-processing"
Sep 12 01:08:59.181054 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.181003    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4eb4187-af72-4a2f-95de-3fd65ae9dd77" containerName="payload-pre-processing"
Sep 12 01:08:59.181054 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.181009    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4eb4187-af72-4a2f-95de-3fd65ae9dd77" containerName="payload-pre-processing"
Sep 12 01:08:59.181054 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.181015    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd4d2e3c-500f-495a-8eb9-331daae91b15" containerName="payload-processing"
Sep 12 01:08:59.181054 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.181020    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd4d2e3c-500f-495a-8eb9-331daae91b15" containerName="payload-processing"
Sep 12 01:08:59.181054 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.181034    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="493f6f55-beab-458e-937e-d42fdf070fb5" containerName="maas-api"
Sep 12 01:08:59.181054 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.181040    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="493f6f55-beab-458e-937e-d42fdf070fb5" containerName="maas-api"
Sep 12 01:08:59.181054 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.181055    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbcd4b65-61de-42b2-a72d-1de5e9443b76" containerName="payload-processing"
Sep 12 01:08:59.181506 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.181063    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbcd4b65-61de-42b2-a72d-1de5e9443b76" containerName="payload-processing"
Sep 12 01:08:59.181506 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.181069    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2ef4545-d14b-4a5c-a2af-80962ed7b545" containerName="payload-pre-processing"
Sep 12 01:08:59.181506 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.181074    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2ef4545-d14b-4a5c-a2af-80962ed7b545" containerName="payload-pre-processing"
Sep 12 01:08:59.181506 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.181146    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2ef4545-d14b-4a5c-a2af-80962ed7b545" containerName="payload-pre-processing"
Sep 12 01:08:59.181506 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.181155    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4eb4187-af72-4a2f-95de-3fd65ae9dd77" containerName="payload-pre-processing"
Sep 12 01:08:59.181506 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.181162    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="15c4992e-a5a3-4069-be73-47ccf19075de" containerName="payload-processing"
Sep 12 01:08:59.181506 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.181168    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7311aa4-3f0a-45c1-a5ba-cf05195cfea8" containerName="payload-processing"
Sep 12 01:08:59.181506 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.181173    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbcd4b65-61de-42b2-a72d-1de5e9443b76" containerName="payload-processing"
Sep 12 01:08:59.181506 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.181179    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9216d8e6-c84e-4651-91e6-41da9378e302" containerName="manager"
Sep 12 01:08:59.181506 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.181184    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd4d2e3c-500f-495a-8eb9-331daae91b15" containerName="payload-processing"
Sep 12 01:08:59.181506 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.181189    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="493f6f55-beab-458e-937e-d42fdf070fb5" containerName="maas-api"
Sep 12 01:08:59.184399 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.184383    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf675f9b4-kvqdh"
Sep 12 01:08:59.188067 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.188046    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 12 01:08:59.188189 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.188103    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 12 01:08:59.188369 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.188352    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 12 01:08:59.188430 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.188387    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5h685\""
Sep 12 01:08:59.191924 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.191714    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cf675f9b4-kvqdh"]
Sep 12 01:08:59.313923 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.313892    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6kw4\" (UniqueName: \"kubernetes.io/projected/2cd35876-322f-48ad-bf3b-cba2d9cefc7b-kube-api-access-f6kw4\") pod \"maas-controller-6cf675f9b4-kvqdh\" (UID: \"2cd35876-322f-48ad-bf3b-cba2d9cefc7b\") " pod="opendatahub/maas-controller-6cf675f9b4-kvqdh"
Sep 12 01:08:59.313923 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.313927    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cd35876-322f-48ad-bf3b-cba2d9cefc7b-metrics-certs\") pod \"maas-controller-6cf675f9b4-kvqdh\" (UID: \"2cd35876-322f-48ad-bf3b-cba2d9cefc7b\") " pod="opendatahub/maas-controller-6cf675f9b4-kvqdh"
Sep 12 01:08:59.314146 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.314060    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2cd35876-322f-48ad-bf3b-cba2d9cefc7b-webhook-cert\") pod \"maas-controller-6cf675f9b4-kvqdh\" (UID: \"2cd35876-322f-48ad-bf3b-cba2d9cefc7b\") " pod="opendatahub/maas-controller-6cf675f9b4-kvqdh"
Sep 12 01:08:59.415379 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.415351    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2cd35876-322f-48ad-bf3b-cba2d9cefc7b-webhook-cert\") pod \"maas-controller-6cf675f9b4-kvqdh\" (UID: \"2cd35876-322f-48ad-bf3b-cba2d9cefc7b\") " pod="opendatahub/maas-controller-6cf675f9b4-kvqdh"
Sep 12 01:08:59.415572 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.415393    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6kw4\" (UniqueName: \"kubernetes.io/projected/2cd35876-322f-48ad-bf3b-cba2d9cefc7b-kube-api-access-f6kw4\") pod \"maas-controller-6cf675f9b4-kvqdh\" (UID: \"2cd35876-322f-48ad-bf3b-cba2d9cefc7b\") " pod="opendatahub/maas-controller-6cf675f9b4-kvqdh"
Sep 12 01:08:59.415572 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.415417    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cd35876-322f-48ad-bf3b-cba2d9cefc7b-metrics-certs\") pod \"maas-controller-6cf675f9b4-kvqdh\" (UID: \"2cd35876-322f-48ad-bf3b-cba2d9cefc7b\") " pod="opendatahub/maas-controller-6cf675f9b4-kvqdh"
Sep 12 01:08:59.418045 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.418012    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2cd35876-322f-48ad-bf3b-cba2d9cefc7b-webhook-cert\") pod \"maas-controller-6cf675f9b4-kvqdh\" (UID: \"2cd35876-322f-48ad-bf3b-cba2d9cefc7b\") " pod="opendatahub/maas-controller-6cf675f9b4-kvqdh"
Sep 12 01:08:59.418254 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.418048    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cd35876-322f-48ad-bf3b-cba2d9cefc7b-metrics-certs\") pod \"maas-controller-6cf675f9b4-kvqdh\" (UID: \"2cd35876-322f-48ad-bf3b-cba2d9cefc7b\") " pod="opendatahub/maas-controller-6cf675f9b4-kvqdh"
Sep 12 01:08:59.424799 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.424779    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6kw4\" (UniqueName: \"kubernetes.io/projected/2cd35876-322f-48ad-bf3b-cba2d9cefc7b-kube-api-access-f6kw4\") pod \"maas-controller-6cf675f9b4-kvqdh\" (UID: \"2cd35876-322f-48ad-bf3b-cba2d9cefc7b\") " pod="opendatahub/maas-controller-6cf675f9b4-kvqdh"
Sep 12 01:08:59.495035 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.494995    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf675f9b4-kvqdh"
Sep 12 01:08:59.769582 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.769553    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cf675f9b4-kvqdh"]
Sep 12 01:08:59.771421 ip-10-0-142-85 kubenswrapper[2558]: W0912 01:08:59.771394    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cd35876_322f_48ad_bf3b_cba2d9cefc7b.slice/crio-918b88e1ff3dd9bc28eaadd22beb8360fdb04eac8422c3a78683a0055995685f WatchSource:0}: Error finding container 918b88e1ff3dd9bc28eaadd22beb8360fdb04eac8422c3a78683a0055995685f: Status 404 returned error can't find the container with id 918b88e1ff3dd9bc28eaadd22beb8360fdb04eac8422c3a78683a0055995685f
Sep 12 01:08:59.772736 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:08:59.772714    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 12 01:09:00.301512 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:00.301476    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf675f9b4-kvqdh" event={"ID":"2cd35876-322f-48ad-bf3b-cba2d9cefc7b","Type":"ContainerStarted","Data":"918b88e1ff3dd9bc28eaadd22beb8360fdb04eac8422c3a78683a0055995685f"}
Sep 12 01:09:01.308923 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:01.308883    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf675f9b4-kvqdh" event={"ID":"2cd35876-322f-48ad-bf3b-cba2d9cefc7b","Type":"ContainerStarted","Data":"e8e945ae7feefff763cf844ab2ae471d34b9e4ba71e3e85373e62e6a39c4a600"}
Sep 12 01:09:01.309416 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:01.309008    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6cf675f9b4-kvqdh"
Sep 12 01:09:01.327070 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:01.327015    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6cf675f9b4-kvqdh" podStartSLOduration=1.718022402 podStartE2EDuration="2.327000256s" totalImagesPullingTime="608.977854ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:08:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:09:01.326061776 +0000 UTC m=+2378.000120635" watchObservedRunningTime="2026-09-12 01:09:01.327000256 +0000 UTC m=+2378.001059114"
Sep 12 01:09:12.316558 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:12.316523    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6cf675f9b4-kvqdh"
Sep 12 01:09:19.432206 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:19.432138    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:09:19.525069 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:19.525033    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6cf675f9b4-kvqdh"]
Sep 12 01:09:19.525305 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:19.525279    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6cf675f9b4-kvqdh" podUID="2cd35876-322f-48ad-bf3b-cba2d9cefc7b" containerName="manager" containerID="cri-o://e8e945ae7feefff763cf844ab2ae471d34b9e4ba71e3e85373e62e6a39c4a600" gracePeriod=30
Sep 12 01:09:19.774497 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:19.774470    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf675f9b4-kvqdh"
Sep 12 01:09:19.816969 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:19.816925    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6kw4\" (UniqueName: \"kubernetes.io/projected/2cd35876-322f-48ad-bf3b-cba2d9cefc7b-kube-api-access-f6kw4\") pod \"2cd35876-322f-48ad-bf3b-cba2d9cefc7b\" (UID: \"2cd35876-322f-48ad-bf3b-cba2d9cefc7b\") "
Sep 12 01:09:19.817166 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:19.817021    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cd35876-322f-48ad-bf3b-cba2d9cefc7b-metrics-certs\") pod \"2cd35876-322f-48ad-bf3b-cba2d9cefc7b\" (UID: \"2cd35876-322f-48ad-bf3b-cba2d9cefc7b\") "
Sep 12 01:09:19.817166 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:19.817065    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2cd35876-322f-48ad-bf3b-cba2d9cefc7b-webhook-cert\") pod \"2cd35876-322f-48ad-bf3b-cba2d9cefc7b\" (UID: \"2cd35876-322f-48ad-bf3b-cba2d9cefc7b\") "
Sep 12 01:09:19.819315 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:19.819284    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2cd35876-322f-48ad-bf3b-cba2d9cefc7b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2cd35876-322f-48ad-bf3b-cba2d9cefc7b" (UID: "2cd35876-322f-48ad-bf3b-cba2d9cefc7b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 01:09:19.819407 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:19.819328    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2cd35876-322f-48ad-bf3b-cba2d9cefc7b-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "2cd35876-322f-48ad-bf3b-cba2d9cefc7b" (UID: "2cd35876-322f-48ad-bf3b-cba2d9cefc7b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 01:09:19.819407 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:19.819343    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cd35876-322f-48ad-bf3b-cba2d9cefc7b-kube-api-access-f6kw4" (OuterVolumeSpecName: "kube-api-access-f6kw4") pod "2cd35876-322f-48ad-bf3b-cba2d9cefc7b" (UID: "2cd35876-322f-48ad-bf3b-cba2d9cefc7b"). InnerVolumeSpecName "kube-api-access-f6kw4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:09:19.918319 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:19.918285    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cd35876-322f-48ad-bf3b-cba2d9cefc7b-metrics-certs\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:09:19.918319 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:19.918313    2558 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2cd35876-322f-48ad-bf3b-cba2d9cefc7b-webhook-cert\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:09:19.918319 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:19.918322    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6kw4\" (UniqueName: \"kubernetes.io/projected/2cd35876-322f-48ad-bf3b-cba2d9cefc7b-kube-api-access-f6kw4\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:09:20.398292 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:20.398255    2558 generic.go:356] "Generic (PLEG): container finished" podID="2cd35876-322f-48ad-bf3b-cba2d9cefc7b" containerID="e8e945ae7feefff763cf844ab2ae471d34b9e4ba71e3e85373e62e6a39c4a600" exitCode=0
Sep 12 01:09:20.398463 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:20.398309    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf675f9b4-kvqdh"
Sep 12 01:09:20.398463 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:20.398335    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf675f9b4-kvqdh" event={"ID":"2cd35876-322f-48ad-bf3b-cba2d9cefc7b","Type":"ContainerDied","Data":"e8e945ae7feefff763cf844ab2ae471d34b9e4ba71e3e85373e62e6a39c4a600"}
Sep 12 01:09:20.398463 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:20.398395    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf675f9b4-kvqdh" event={"ID":"2cd35876-322f-48ad-bf3b-cba2d9cefc7b","Type":"ContainerDied","Data":"918b88e1ff3dd9bc28eaadd22beb8360fdb04eac8422c3a78683a0055995685f"}
Sep 12 01:09:20.398463 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:20.398427    2558 scope.go:117] "RemoveContainer" containerID="e8e945ae7feefff763cf844ab2ae471d34b9e4ba71e3e85373e62e6a39c4a600"
Sep 12 01:09:20.407528 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:20.407510    2558 scope.go:117] "RemoveContainer" containerID="e8e945ae7feefff763cf844ab2ae471d34b9e4ba71e3e85373e62e6a39c4a600"
Sep 12 01:09:20.407812 ip-10-0-142-85 kubenswrapper[2558]: E0912 01:09:20.407792    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8e945ae7feefff763cf844ab2ae471d34b9e4ba71e3e85373e62e6a39c4a600\": container with ID starting with e8e945ae7feefff763cf844ab2ae471d34b9e4ba71e3e85373e62e6a39c4a600 not found: ID does not exist" containerID="e8e945ae7feefff763cf844ab2ae471d34b9e4ba71e3e85373e62e6a39c4a600"
Sep 12 01:09:20.407879 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:20.407818    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8e945ae7feefff763cf844ab2ae471d34b9e4ba71e3e85373e62e6a39c4a600"} err="failed to get container status \"e8e945ae7feefff763cf844ab2ae471d34b9e4ba71e3e85373e62e6a39c4a600\": rpc error: code = NotFound desc = could not find container \"e8e945ae7feefff763cf844ab2ae471d34b9e4ba71e3e85373e62e6a39c4a600\": container with ID starting with e8e945ae7feefff763cf844ab2ae471d34b9e4ba71e3e85373e62e6a39c4a600 not found: ID does not exist"
Sep 12 01:09:20.419340 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:20.419320    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6cf675f9b4-kvqdh"]
Sep 12 01:09:20.420823 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:20.420804    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6cf675f9b4-kvqdh"]
Sep 12 01:09:21.071850 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:21.071816    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:09:21.184135 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:21.184098    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6cf675f9b4-jr6fr"]
Sep 12 01:09:21.184516 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:21.184500    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cd35876-322f-48ad-bf3b-cba2d9cefc7b" containerName="manager"
Sep 12 01:09:21.184563 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:21.184519    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cd35876-322f-48ad-bf3b-cba2d9cefc7b" containerName="manager"
Sep 12 01:09:21.184595 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:21.184586    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cd35876-322f-48ad-bf3b-cba2d9cefc7b" containerName="manager"
Sep 12 01:09:21.188613 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:21.188595    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf675f9b4-jr6fr"
Sep 12 01:09:21.192182 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:21.192160    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 12 01:09:21.192316 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:21.192216    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 12 01:09:21.192316 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:21.192249    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 12 01:09:21.192316 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:21.192218    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5h685\""
Sep 12 01:09:21.197056 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:21.197030    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cf675f9b4-jr6fr"]
Sep 12 01:09:21.229779 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:21.229752    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/342bbca2-a420-461f-be92-fa68b01c0f31-metrics-certs\") pod \"maas-controller-6cf675f9b4-jr6fr\" (UID: \"342bbca2-a420-461f-be92-fa68b01c0f31\") " pod="opendatahub/maas-controller-6cf675f9b4-jr6fr"
Sep 12 01:09:21.229912 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:21.229814    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/342bbca2-a420-461f-be92-fa68b01c0f31-webhook-cert\") pod \"maas-controller-6cf675f9b4-jr6fr\" (UID: \"342bbca2-a420-461f-be92-fa68b01c0f31\") " pod="opendatahub/maas-controller-6cf675f9b4-jr6fr"
Sep 12 01:09:21.229912 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:21.229876    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrjrk\" (UniqueName: \"kubernetes.io/projected/342bbca2-a420-461f-be92-fa68b01c0f31-kube-api-access-rrjrk\") pod \"maas-controller-6cf675f9b4-jr6fr\" (UID: \"342bbca2-a420-461f-be92-fa68b01c0f31\") " pod="opendatahub/maas-controller-6cf675f9b4-jr6fr"
Sep 12 01:09:21.331190 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:21.331051    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/342bbca2-a420-461f-be92-fa68b01c0f31-metrics-certs\") pod \"maas-controller-6cf675f9b4-jr6fr\" (UID: \"342bbca2-a420-461f-be92-fa68b01c0f31\") " pod="opendatahub/maas-controller-6cf675f9b4-jr6fr"
Sep 12 01:09:21.331320 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:21.331210    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/342bbca2-a420-461f-be92-fa68b01c0f31-webhook-cert\") pod \"maas-controller-6cf675f9b4-jr6fr\" (UID: \"342bbca2-a420-461f-be92-fa68b01c0f31\") " pod="opendatahub/maas-controller-6cf675f9b4-jr6fr"
Sep 12 01:09:21.331320 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:21.331239    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrjrk\" (UniqueName: \"kubernetes.io/projected/342bbca2-a420-461f-be92-fa68b01c0f31-kube-api-access-rrjrk\") pod \"maas-controller-6cf675f9b4-jr6fr\" (UID: \"342bbca2-a420-461f-be92-fa68b01c0f31\") " pod="opendatahub/maas-controller-6cf675f9b4-jr6fr"
Sep 12 01:09:21.333898 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:21.333865    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/342bbca2-a420-461f-be92-fa68b01c0f31-webhook-cert\") pod \"maas-controller-6cf675f9b4-jr6fr\" (UID: \"342bbca2-a420-461f-be92-fa68b01c0f31\") " pod="opendatahub/maas-controller-6cf675f9b4-jr6fr"
Sep 12 01:09:21.334018 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:21.333879    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/342bbca2-a420-461f-be92-fa68b01c0f31-metrics-certs\") pod \"maas-controller-6cf675f9b4-jr6fr\" (UID: \"342bbca2-a420-461f-be92-fa68b01c0f31\") " pod="opendatahub/maas-controller-6cf675f9b4-jr6fr"
Sep 12 01:09:21.339624 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:21.339597    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrjrk\" (UniqueName: \"kubernetes.io/projected/342bbca2-a420-461f-be92-fa68b01c0f31-kube-api-access-rrjrk\") pod \"maas-controller-6cf675f9b4-jr6fr\" (UID: \"342bbca2-a420-461f-be92-fa68b01c0f31\") " pod="opendatahub/maas-controller-6cf675f9b4-jr6fr"
Sep 12 01:09:21.499651 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:21.499613    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf675f9b4-jr6fr"
Sep 12 01:09:21.741239 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:21.739511    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2cd35876-322f-48ad-bf3b-cba2d9cefc7b" path="/var/lib/kubelet/pods/2cd35876-322f-48ad-bf3b-cba2d9cefc7b/volumes"
Sep 12 01:09:21.772171 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:21.772145    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cf675f9b4-jr6fr"]
Sep 12 01:09:21.774278 ip-10-0-142-85 kubenswrapper[2558]: W0912 01:09:21.774250    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod342bbca2_a420_461f_be92_fa68b01c0f31.slice/crio-7a6740938223fbf1355bf5b1bde83cc476ac1ee0c56927d0c6dd1fc673620d26 WatchSource:0}: Error finding container 7a6740938223fbf1355bf5b1bde83cc476ac1ee0c56927d0c6dd1fc673620d26: Status 404 returned error can't find the container with id 7a6740938223fbf1355bf5b1bde83cc476ac1ee0c56927d0c6dd1fc673620d26
Sep 12 01:09:22.411941 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:22.411847    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf675f9b4-jr6fr" event={"ID":"342bbca2-a420-461f-be92-fa68b01c0f31","Type":"ContainerStarted","Data":"0d23ae40fd1fc1b2e46b1769ff9ee342d23b68e76ba65c867077966a6a3c1496"}
Sep 12 01:09:22.411941 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:22.411905    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf675f9b4-jr6fr" event={"ID":"342bbca2-a420-461f-be92-fa68b01c0f31","Type":"ContainerStarted","Data":"7a6740938223fbf1355bf5b1bde83cc476ac1ee0c56927d0c6dd1fc673620d26"}
Sep 12 01:09:22.412392 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:22.411995    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6cf675f9b4-jr6fr"
Sep 12 01:09:22.431999 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:22.431943    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6cf675f9b4-jr6fr" podStartSLOduration=1.103820404 podStartE2EDuration="1.431921093s" totalImagesPullingTime="328.100689ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:09:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:09:22.431111142 +0000 UTC m=+2399.105169996" watchObservedRunningTime="2026-09-12 01:09:22.431921093 +0000 UTC m=+2399.105979952"
Sep 12 01:09:23.926575 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:23.926460    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-czf8h_dd90996a-5428-4ccf-a154-c0aa8def9789/console-operator/1.log"
Sep 12 01:09:23.951860 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:23.939861    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-czf8h_dd90996a-5428-4ccf-a154-c0aa8def9789/console-operator/1.log"
Sep 12 01:09:33.259166 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:33.259131    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:09:33.319128 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:33.319072    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:09:33.419294 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:09:33.419259    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6cf675f9b4-jr6fr"
Sep 12 01:10:24.769167 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:10:24.769134    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:10:33.576775 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:10:33.576738    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:10:33.631486 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:10:33.631453    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:10:57.144019 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:10:57.143988    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:10:57.244838 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:10:57.244801    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:11:03.321888 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:11:03.321847    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:12:26.456986 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:12:26.456934    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:12:42.022408 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:12:42.022369    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449"]
Sep 12 01:12:42.026571 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:12:42.026541    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449"
Sep 12 01:12:42.029276 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:12:42.029247    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-452393ea-kserve-self-signed-certs\""
Sep 12 01:12:42.036026 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:12:42.035998    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449"]
Sep 12 01:12:42.053016 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:12:42.052982    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:12:42.124645 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:12:42.124607    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f15649f9-5f15-4e59-8c9e-c15da8833a09-dshm\") pod \"e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449\" (UID: \"f15649f9-5f15-4e59-8c9e-c15da8833a09\") " pod="llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449"
Sep 12 01:12:42.124863 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:12:42.124658    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f15649f9-5f15-4e59-8c9e-c15da8833a09-model-cache\") pod \"e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449\" (UID: \"f15649f9-5f15-4e59-8c9e-c15da8833a09\") " pod="llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449"
Sep 12 01:12:42.124863 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:12:42.124740    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f15649f9-5f15-4e59-8c9e-c15da8833a09-tmp-dir\") pod \"e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449\" (UID: \"f15649f9-5f15-4e59-8c9e-c15da8833a09\") " pod="llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449"
Sep 12 01:12:42.124863 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:12:42.124778    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f15649f9-5f15-4e59-8c9e-c15da8833a09-tls-certs\") pod \"e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449\" (UID: \"f15649f9-5f15-4e59-8c9e-c15da8833a09\") " pod="llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449"
Sep 12 01:12:42.124863 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:12:42.124853    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f15649f9-5f15-4e59-8c9e-c15da8833a09-home\") pod \"e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449\" (UID: \"f15649f9-5f15-4e59-8c9e-c15da8833a09\") " pod="llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449"
Sep 12 01:12:42.125036 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:12:42.124913    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgwsk\" (UniqueName: \"kubernetes.io/projected/f15649f9-5f15-4e59-8c9e-c15da8833a09-kube-api-access-qgwsk\") pod \"e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449\" (UID: \"f15649f9-5f15-4e59-8c9e-c15da8833a09\") " pod="llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449"
Sep 12 01:12:42.225532 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:12:42.225499    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgwsk\" (UniqueName: \"kubernetes.io/projected/f15649f9-5f15-4e59-8c9e-c15da8833a09-kube-api-access-qgwsk\") pod \"e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449\" (UID: \"f15649f9-5f15-4e59-8c9e-c15da8833a09\") " pod="llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449"
Sep 12 01:12:42.225532 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:12:42.225539    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f15649f9-5f15-4e59-8c9e-c15da8833a09-dshm\") pod \"e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449\" (UID: \"f15649f9-5f15-4e59-8c9e-c15da8833a09\") " pod="llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449"
Sep 12 01:12:42.225813 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:12:42.225569    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f15649f9-5f15-4e59-8c9e-c15da8833a09-model-cache\") pod \"e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449\" (UID: \"f15649f9-5f15-4e59-8c9e-c15da8833a09\") " pod="llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449"
Sep 12 01:12:42.225813 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:12:42.225603    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f15649f9-5f15-4e59-8c9e-c15da8833a09-tmp-dir\") pod \"e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449\" (UID: \"f15649f9-5f15-4e59-8c9e-c15da8833a09\") " pod="llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449"
Sep 12 01:12:42.225813 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:12:42.225636    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f15649f9-5f15-4e59-8c9e-c15da8833a09-tls-certs\") pod \"e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449\" (UID: \"f15649f9-5f15-4e59-8c9e-c15da8833a09\") " pod="llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449"
Sep 12 01:12:42.225813 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:12:42.225674    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f15649f9-5f15-4e59-8c9e-c15da8833a09-home\") pod \"e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449\" (UID: \"f15649f9-5f15-4e59-8c9e-c15da8833a09\") " pod="llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449"
Sep 12 01:12:42.226115 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:12:42.226060    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f15649f9-5f15-4e59-8c9e-c15da8833a09-model-cache\") pod \"e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449\" (UID: \"f15649f9-5f15-4e59-8c9e-c15da8833a09\") " pod="llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449"
Sep 12 01:12:42.226193 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:12:42.226124    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f15649f9-5f15-4e59-8c9e-c15da8833a09-home\") pod \"e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449\" (UID: \"f15649f9-5f15-4e59-8c9e-c15da8833a09\") " pod="llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449"
Sep 12 01:12:42.226193 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:12:42.226124    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f15649f9-5f15-4e59-8c9e-c15da8833a09-tmp-dir\") pod \"e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449\" (UID: \"f15649f9-5f15-4e59-8c9e-c15da8833a09\") " pod="llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449"
Sep 12 01:12:42.228149 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:12:42.228118    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f15649f9-5f15-4e59-8c9e-c15da8833a09-dshm\") pod \"e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449\" (UID: \"f15649f9-5f15-4e59-8c9e-c15da8833a09\") " pod="llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449"
Sep 12 01:12:42.228395 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:12:42.228380    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f15649f9-5f15-4e59-8c9e-c15da8833a09-tls-certs\") pod \"e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449\" (UID: \"f15649f9-5f15-4e59-8c9e-c15da8833a09\") " pod="llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449"
Sep 12 01:12:42.235228 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:12:42.235205    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgwsk\" (UniqueName: \"kubernetes.io/projected/f15649f9-5f15-4e59-8c9e-c15da8833a09-kube-api-access-qgwsk\") pod \"e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449\" (UID: \"f15649f9-5f15-4e59-8c9e-c15da8833a09\") " pod="llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449"
Sep 12 01:12:42.337419 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:12:42.337313    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449"
Sep 12 01:12:42.611927 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:12:42.611902    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449"]
Sep 12 01:12:42.614647 ip-10-0-142-85 kubenswrapper[2558]: W0912 01:12:42.614610    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf15649f9_5f15_4e59_8c9e_c15da8833a09.slice/crio-0bc5efdc8d7a5c2bf4cddf5156b2a76a2b78525c76d789b5613bfc8a078ab7bb WatchSource:0}: Error finding container 0bc5efdc8d7a5c2bf4cddf5156b2a76a2b78525c76d789b5613bfc8a078ab7bb: Status 404 returned error can't find the container with id 0bc5efdc8d7a5c2bf4cddf5156b2a76a2b78525c76d789b5613bfc8a078ab7bb
Sep 12 01:12:43.358574 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:12:43.358537    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449" event={"ID":"f15649f9-5f15-4e59-8c9e-c15da8833a09","Type":"ContainerStarted","Data":"16587995b1c8001f77235e85530533ea4742ede2f1a09f1e5f4891d4f02f35fb"}
Sep 12 01:12:43.358574 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:12:43.358579    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449" event={"ID":"f15649f9-5f15-4e59-8c9e-c15da8833a09","Type":"ContainerStarted","Data":"0bc5efdc8d7a5c2bf4cddf5156b2a76a2b78525c76d789b5613bfc8a078ab7bb"}
Sep 12 01:12:43.375310 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:12:43.375264    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449" podStartSLOduration=1.3752484790000001 podStartE2EDuration="1.375248479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:12:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:12:43.374222545 +0000 UTC m=+2600.048281405" watchObservedRunningTime="2026-09-12 01:12:43.375248479 +0000 UTC m=+2600.049307337"
Sep 12 01:12:52.338193 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:12:52.338161    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449"
Sep 12 01:12:52.338674 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:12:52.338210    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449"
Sep 12 01:12:52.350728 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:12:52.350704    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449"
Sep 12 01:12:52.412222 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:12:52.412193    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449"
Sep 12 01:12:55.855753 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:12:55.855709    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:13:05.964298 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:05.964258    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:13:07.971740 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:07.971704    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449"]
Sep 12 01:13:07.992470 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:07.992441    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:13:22.993065 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:22.993021    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449" podUID="f15649f9-5f15-4e59-8c9e-c15da8833a09" containerName="main" containerID="cri-o://16587995b1c8001f77235e85530533ea4742ede2f1a09f1e5f4891d4f02f35fb" gracePeriod=45
Sep 12 01:13:23.250106 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:23.250012    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449"
Sep 12 01:13:23.316700 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:23.316667    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qgwsk\" (UniqueName: \"kubernetes.io/projected/f15649f9-5f15-4e59-8c9e-c15da8833a09-kube-api-access-qgwsk\") pod \"f15649f9-5f15-4e59-8c9e-c15da8833a09\" (UID: \"f15649f9-5f15-4e59-8c9e-c15da8833a09\") "
Sep 12 01:13:23.316879 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:23.316748    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f15649f9-5f15-4e59-8c9e-c15da8833a09-dshm\") pod \"f15649f9-5f15-4e59-8c9e-c15da8833a09\" (UID: \"f15649f9-5f15-4e59-8c9e-c15da8833a09\") "
Sep 12 01:13:23.316879 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:23.316788    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f15649f9-5f15-4e59-8c9e-c15da8833a09-tmp-dir\") pod \"f15649f9-5f15-4e59-8c9e-c15da8833a09\" (UID: \"f15649f9-5f15-4e59-8c9e-c15da8833a09\") "
Sep 12 01:13:23.316879 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:23.316831    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f15649f9-5f15-4e59-8c9e-c15da8833a09-model-cache\") pod \"f15649f9-5f15-4e59-8c9e-c15da8833a09\" (UID: \"f15649f9-5f15-4e59-8c9e-c15da8833a09\") "
Sep 12 01:13:23.316879 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:23.316855    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f15649f9-5f15-4e59-8c9e-c15da8833a09-tls-certs\") pod \"f15649f9-5f15-4e59-8c9e-c15da8833a09\" (UID: \"f15649f9-5f15-4e59-8c9e-c15da8833a09\") "
Sep 12 01:13:23.316879 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:23.316878    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f15649f9-5f15-4e59-8c9e-c15da8833a09-home\") pod \"f15649f9-5f15-4e59-8c9e-c15da8833a09\" (UID: \"f15649f9-5f15-4e59-8c9e-c15da8833a09\") "
Sep 12 01:13:23.317141 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:23.317067    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f15649f9-5f15-4e59-8c9e-c15da8833a09-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f15649f9-5f15-4e59-8c9e-c15da8833a09" (UID: "f15649f9-5f15-4e59-8c9e-c15da8833a09"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:13:23.317141 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:23.317121    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f15649f9-5f15-4e59-8c9e-c15da8833a09-model-cache" (OuterVolumeSpecName: "model-cache") pod "f15649f9-5f15-4e59-8c9e-c15da8833a09" (UID: "f15649f9-5f15-4e59-8c9e-c15da8833a09"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:13:23.317314 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:23.317285    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f15649f9-5f15-4e59-8c9e-c15da8833a09-home" (OuterVolumeSpecName: "home") pod "f15649f9-5f15-4e59-8c9e-c15da8833a09" (UID: "f15649f9-5f15-4e59-8c9e-c15da8833a09"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:13:23.319192 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:23.319157    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f15649f9-5f15-4e59-8c9e-c15da8833a09-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f15649f9-5f15-4e59-8c9e-c15da8833a09" (UID: "f15649f9-5f15-4e59-8c9e-c15da8833a09"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 01:13:23.319192 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:23.319182    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f15649f9-5f15-4e59-8c9e-c15da8833a09-dshm" (OuterVolumeSpecName: "dshm") pod "f15649f9-5f15-4e59-8c9e-c15da8833a09" (UID: "f15649f9-5f15-4e59-8c9e-c15da8833a09"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 12 01:13:23.319326 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:23.319198    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f15649f9-5f15-4e59-8c9e-c15da8833a09-kube-api-access-qgwsk" (OuterVolumeSpecName: "kube-api-access-qgwsk") pod "f15649f9-5f15-4e59-8c9e-c15da8833a09" (UID: "f15649f9-5f15-4e59-8c9e-c15da8833a09"). InnerVolumeSpecName "kube-api-access-qgwsk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:13:23.417800 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:23.417770    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f15649f9-5f15-4e59-8c9e-c15da8833a09-model-cache\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:13:23.417800 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:23.417798    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f15649f9-5f15-4e59-8c9e-c15da8833a09-tls-certs\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:13:23.417800 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:23.417805    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f15649f9-5f15-4e59-8c9e-c15da8833a09-home\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:13:23.418042 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:23.417813    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgwsk\" (UniqueName: \"kubernetes.io/projected/f15649f9-5f15-4e59-8c9e-c15da8833a09-kube-api-access-qgwsk\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:13:23.418042 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:23.417822    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f15649f9-5f15-4e59-8c9e-c15da8833a09-dshm\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:13:23.418042 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:23.417852    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f15649f9-5f15-4e59-8c9e-c15da8833a09-tmp-dir\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:13:23.547721 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:23.547633    2558 generic.go:356] "Generic (PLEG): container finished" podID="f15649f9-5f15-4e59-8c9e-c15da8833a09" containerID="16587995b1c8001f77235e85530533ea4742ede2f1a09f1e5f4891d4f02f35fb" exitCode=0
Sep 12 01:13:23.547721 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:23.547677    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449"
Sep 12 01:13:23.547721 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:23.547707    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449" event={"ID":"f15649f9-5f15-4e59-8c9e-c15da8833a09","Type":"ContainerDied","Data":"16587995b1c8001f77235e85530533ea4742ede2f1a09f1e5f4891d4f02f35fb"}
Sep 12 01:13:23.547960 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:23.547739    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449" event={"ID":"f15649f9-5f15-4e59-8c9e-c15da8833a09","Type":"ContainerDied","Data":"0bc5efdc8d7a5c2bf4cddf5156b2a76a2b78525c76d789b5613bfc8a078ab7bb"}
Sep 12 01:13:23.547960 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:23.547756    2558 scope.go:117] "RemoveContainer" containerID="16587995b1c8001f77235e85530533ea4742ede2f1a09f1e5f4891d4f02f35fb"
Sep 12 01:13:23.556879 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:23.556858    2558 scope.go:117] "RemoveContainer" containerID="16587995b1c8001f77235e85530533ea4742ede2f1a09f1e5f4891d4f02f35fb"
Sep 12 01:13:23.557151 ip-10-0-142-85 kubenswrapper[2558]: E0912 01:13:23.557127    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16587995b1c8001f77235e85530533ea4742ede2f1a09f1e5f4891d4f02f35fb\": container with ID starting with 16587995b1c8001f77235e85530533ea4742ede2f1a09f1e5f4891d4f02f35fb not found: ID does not exist" containerID="16587995b1c8001f77235e85530533ea4742ede2f1a09f1e5f4891d4f02f35fb"
Sep 12 01:13:23.557246 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:23.557158    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16587995b1c8001f77235e85530533ea4742ede2f1a09f1e5f4891d4f02f35fb"} err="failed to get container status \"16587995b1c8001f77235e85530533ea4742ede2f1a09f1e5f4891d4f02f35fb\": rpc error: code = NotFound desc = could not find container \"16587995b1c8001f77235e85530533ea4742ede2f1a09f1e5f4891d4f02f35fb\": container with ID starting with 16587995b1c8001f77235e85530533ea4742ede2f1a09f1e5f4891d4f02f35fb not found: ID does not exist"
Sep 12 01:13:23.567893 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:23.567871    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449"]
Sep 12 01:13:23.573631 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:23.573611    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-452393ea-kserve-65bd44cdbc-vx449"]
Sep 12 01:13:23.738804 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:23.738773    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f15649f9-5f15-4e59-8c9e-c15da8833a09" path="/var/lib/kubelet/pods/f15649f9-5f15-4e59-8c9e-c15da8833a09/volumes"
Sep 12 01:13:34.164029 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:34.163997    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-65b566db79-tjbz4"]
Sep 12 01:13:34.164628 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:34.164609    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f15649f9-5f15-4e59-8c9e-c15da8833a09" containerName="main"
Sep 12 01:13:34.164713 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:34.164630    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f15649f9-5f15-4e59-8c9e-c15da8833a09" containerName="main"
Sep 12 01:13:34.164768 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:34.164719    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f15649f9-5f15-4e59-8c9e-c15da8833a09" containerName="main"
Sep 12 01:13:34.167563 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:34.167534    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-65b566db79-tjbz4"
Sep 12 01:13:34.179994 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:34.179970    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-65b566db79-tjbz4"]
Sep 12 01:13:34.216457 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:34.216426    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/011c92b3-f7c0-46c5-bceb-ddbb69a30c13-openshift-service-ca\") pod \"maas-api-65b566db79-tjbz4\" (UID: \"011c92b3-f7c0-46c5-bceb-ddbb69a30c13\") " pod="odh-ai-gateway-infra/maas-api-65b566db79-tjbz4"
Sep 12 01:13:34.216457 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:34.216464    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qks6x\" (UniqueName: \"kubernetes.io/projected/011c92b3-f7c0-46c5-bceb-ddbb69a30c13-kube-api-access-qks6x\") pod \"maas-api-65b566db79-tjbz4\" (UID: \"011c92b3-f7c0-46c5-bceb-ddbb69a30c13\") " pod="odh-ai-gateway-infra/maas-api-65b566db79-tjbz4"
Sep 12 01:13:34.216673 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:34.216558    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/011c92b3-f7c0-46c5-bceb-ddbb69a30c13-maas-api-tls\") pod \"maas-api-65b566db79-tjbz4\" (UID: \"011c92b3-f7c0-46c5-bceb-ddbb69a30c13\") " pod="odh-ai-gateway-infra/maas-api-65b566db79-tjbz4"
Sep 12 01:13:34.216673 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:34.216610    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/011c92b3-f7c0-46c5-bceb-ddbb69a30c13-metrics-certs\") pod \"maas-api-65b566db79-tjbz4\" (UID: \"011c92b3-f7c0-46c5-bceb-ddbb69a30c13\") " pod="odh-ai-gateway-infra/maas-api-65b566db79-tjbz4"
Sep 12 01:13:34.318111 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:34.318057    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/011c92b3-f7c0-46c5-bceb-ddbb69a30c13-maas-api-tls\") pod \"maas-api-65b566db79-tjbz4\" (UID: \"011c92b3-f7c0-46c5-bceb-ddbb69a30c13\") " pod="odh-ai-gateway-infra/maas-api-65b566db79-tjbz4"
Sep 12 01:13:34.318318 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:34.318136    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/011c92b3-f7c0-46c5-bceb-ddbb69a30c13-metrics-certs\") pod \"maas-api-65b566db79-tjbz4\" (UID: \"011c92b3-f7c0-46c5-bceb-ddbb69a30c13\") " pod="odh-ai-gateway-infra/maas-api-65b566db79-tjbz4"
Sep 12 01:13:34.318318 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:34.318291    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/011c92b3-f7c0-46c5-bceb-ddbb69a30c13-openshift-service-ca\") pod \"maas-api-65b566db79-tjbz4\" (UID: \"011c92b3-f7c0-46c5-bceb-ddbb69a30c13\") " pod="odh-ai-gateway-infra/maas-api-65b566db79-tjbz4"
Sep 12 01:13:34.318437 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:34.318327    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qks6x\" (UniqueName: \"kubernetes.io/projected/011c92b3-f7c0-46c5-bceb-ddbb69a30c13-kube-api-access-qks6x\") pod \"maas-api-65b566db79-tjbz4\" (UID: \"011c92b3-f7c0-46c5-bceb-ddbb69a30c13\") " pod="odh-ai-gateway-infra/maas-api-65b566db79-tjbz4"
Sep 12 01:13:34.318986 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:34.318952    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/011c92b3-f7c0-46c5-bceb-ddbb69a30c13-openshift-service-ca\") pod \"maas-api-65b566db79-tjbz4\" (UID: \"011c92b3-f7c0-46c5-bceb-ddbb69a30c13\") " pod="odh-ai-gateway-infra/maas-api-65b566db79-tjbz4"
Sep 12 01:13:34.320765 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:34.320737    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/011c92b3-f7c0-46c5-bceb-ddbb69a30c13-metrics-certs\") pod \"maas-api-65b566db79-tjbz4\" (UID: \"011c92b3-f7c0-46c5-bceb-ddbb69a30c13\") " pod="odh-ai-gateway-infra/maas-api-65b566db79-tjbz4"
Sep 12 01:13:34.320882 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:34.320831    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/011c92b3-f7c0-46c5-bceb-ddbb69a30c13-maas-api-tls\") pod \"maas-api-65b566db79-tjbz4\" (UID: \"011c92b3-f7c0-46c5-bceb-ddbb69a30c13\") " pod="odh-ai-gateway-infra/maas-api-65b566db79-tjbz4"
Sep 12 01:13:34.326694 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:34.326672    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qks6x\" (UniqueName: \"kubernetes.io/projected/011c92b3-f7c0-46c5-bceb-ddbb69a30c13-kube-api-access-qks6x\") pod \"maas-api-65b566db79-tjbz4\" (UID: \"011c92b3-f7c0-46c5-bceb-ddbb69a30c13\") " pod="odh-ai-gateway-infra/maas-api-65b566db79-tjbz4"
Sep 12 01:13:34.479690 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:34.479652    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-65b566db79-tjbz4"
Sep 12 01:13:34.755681 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:34.755656    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-65b566db79-tjbz4"]
Sep 12 01:13:34.758437 ip-10-0-142-85 kubenswrapper[2558]: W0912 01:13:34.758404    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod011c92b3_f7c0_46c5_bceb_ddbb69a30c13.slice/crio-06f0f54ecffb05dd5700413c8312a0c2503027992ea4f7e2cf99f77778ee1bde WatchSource:0}: Error finding container 06f0f54ecffb05dd5700413c8312a0c2503027992ea4f7e2cf99f77778ee1bde: Status 404 returned error can't find the container with id 06f0f54ecffb05dd5700413c8312a0c2503027992ea4f7e2cf99f77778ee1bde
Sep 12 01:13:35.606691 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:35.606655    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-65b566db79-tjbz4" event={"ID":"011c92b3-f7c0-46c5-bceb-ddbb69a30c13","Type":"ContainerStarted","Data":"4df9295971301f5950ad94a6560bf7fbb39772ab4b631294f88c587118269f46"}
Sep 12 01:13:35.607118 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:35.606707    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-65b566db79-tjbz4" event={"ID":"011c92b3-f7c0-46c5-bceb-ddbb69a30c13","Type":"ContainerStarted","Data":"06f0f54ecffb05dd5700413c8312a0c2503027992ea4f7e2cf99f77778ee1bde"}
Sep 12 01:13:35.607118 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:35.606737    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-65b566db79-tjbz4"
Sep 12 01:13:35.625747 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:35.625693    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-65b566db79-tjbz4" podStartSLOduration=1.275239446 podStartE2EDuration="1.625664051s" totalImagesPullingTime="350.424605ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-12 01:13:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:13:35.624869114 +0000 UTC m=+2652.298927974" watchObservedRunningTime="2026-09-12 01:13:35.625664051 +0000 UTC m=+2652.299722968"
Sep 12 01:13:41.617541 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:41.617511    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-65b566db79-tjbz4"
Sep 12 01:13:41.663608 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:41.663568    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-84484d59ff-fqvpv"]
Sep 12 01:13:41.663855 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:41.663829    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-84484d59ff-fqvpv" podUID="0cf21cd5-aa7b-47a8-b2da-137f8b53f373" containerName="maas-api" containerID="cri-o://c2f64ace921f17cfe4c81354a6c2fc5646e04575674a3e0e89f5f3a7b53c2b0b" gracePeriod=30
Sep 12 01:13:42.910940 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:42.910915    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-84484d59ff-fqvpv"
Sep 12 01:13:43.007743 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:43.007650    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cf21cd5-aa7b-47a8-b2da-137f8b53f373-metrics-certs\") pod \"0cf21cd5-aa7b-47a8-b2da-137f8b53f373\" (UID: \"0cf21cd5-aa7b-47a8-b2da-137f8b53f373\") "
Sep 12 01:13:43.007743 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:43.007699    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cf21cd5-aa7b-47a8-b2da-137f8b53f373-openshift-service-ca\") pod \"0cf21cd5-aa7b-47a8-b2da-137f8b53f373\" (UID: \"0cf21cd5-aa7b-47a8-b2da-137f8b53f373\") "
Sep 12 01:13:43.007743 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:43.007735    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cf21cd5-aa7b-47a8-b2da-137f8b53f373-maas-api-tls\") pod \"0cf21cd5-aa7b-47a8-b2da-137f8b53f373\" (UID: \"0cf21cd5-aa7b-47a8-b2da-137f8b53f373\") "
Sep 12 01:13:43.007978 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:43.007774    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jvfr9\" (UniqueName: \"kubernetes.io/projected/0cf21cd5-aa7b-47a8-b2da-137f8b53f373-kube-api-access-jvfr9\") pod \"0cf21cd5-aa7b-47a8-b2da-137f8b53f373\" (UID: \"0cf21cd5-aa7b-47a8-b2da-137f8b53f373\") "
Sep 12 01:13:43.008206 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:43.008183    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cf21cd5-aa7b-47a8-b2da-137f8b53f373-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0cf21cd5-aa7b-47a8-b2da-137f8b53f373" (UID: "0cf21cd5-aa7b-47a8-b2da-137f8b53f373"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 12 01:13:43.010223 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:43.010195    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0cf21cd5-aa7b-47a8-b2da-137f8b53f373-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0cf21cd5-aa7b-47a8-b2da-137f8b53f373" (UID: "0cf21cd5-aa7b-47a8-b2da-137f8b53f373"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 01:13:43.010325 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:43.010221    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0cf21cd5-aa7b-47a8-b2da-137f8b53f373-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0cf21cd5-aa7b-47a8-b2da-137f8b53f373" (UID: "0cf21cd5-aa7b-47a8-b2da-137f8b53f373"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 12 01:13:43.010325 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:43.010252    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cf21cd5-aa7b-47a8-b2da-137f8b53f373-kube-api-access-jvfr9" (OuterVolumeSpecName: "kube-api-access-jvfr9") pod "0cf21cd5-aa7b-47a8-b2da-137f8b53f373" (UID: "0cf21cd5-aa7b-47a8-b2da-137f8b53f373"). InnerVolumeSpecName "kube-api-access-jvfr9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 12 01:13:43.109141 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:43.109105    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cf21cd5-aa7b-47a8-b2da-137f8b53f373-openshift-service-ca\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:13:43.109141 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:43.109134    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cf21cd5-aa7b-47a8-b2da-137f8b53f373-maas-api-tls\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:13:43.109141 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:43.109143    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvfr9\" (UniqueName: \"kubernetes.io/projected/0cf21cd5-aa7b-47a8-b2da-137f8b53f373-kube-api-access-jvfr9\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:13:43.109386 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:43.109152    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cf21cd5-aa7b-47a8-b2da-137f8b53f373-metrics-certs\") on node \"ip-10-0-142-85.ec2.internal\" DevicePath \"\""
Sep 12 01:13:43.645786 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:43.645756    2558 generic.go:356] "Generic (PLEG): container finished" podID="0cf21cd5-aa7b-47a8-b2da-137f8b53f373" containerID="c2f64ace921f17cfe4c81354a6c2fc5646e04575674a3e0e89f5f3a7b53c2b0b" exitCode=0
Sep 12 01:13:43.645976 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:43.645805    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-84484d59ff-fqvpv"
Sep 12 01:13:43.645976 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:43.645811    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-84484d59ff-fqvpv" event={"ID":"0cf21cd5-aa7b-47a8-b2da-137f8b53f373","Type":"ContainerDied","Data":"c2f64ace921f17cfe4c81354a6c2fc5646e04575674a3e0e89f5f3a7b53c2b0b"}
Sep 12 01:13:43.645976 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:43.645913    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-84484d59ff-fqvpv" event={"ID":"0cf21cd5-aa7b-47a8-b2da-137f8b53f373","Type":"ContainerDied","Data":"450cafc8d6f4ce3019851d2c5bb15507f3e38ad5822254ea8082a6f2cb8aa7a5"}
Sep 12 01:13:43.645976 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:43.645937    2558 scope.go:117] "RemoveContainer" containerID="c2f64ace921f17cfe4c81354a6c2fc5646e04575674a3e0e89f5f3a7b53c2b0b"
Sep 12 01:13:43.655819 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:43.655801    2558 scope.go:117] "RemoveContainer" containerID="c2f64ace921f17cfe4c81354a6c2fc5646e04575674a3e0e89f5f3a7b53c2b0b"
Sep 12 01:13:43.656102 ip-10-0-142-85 kubenswrapper[2558]: E0912 01:13:43.656065    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2f64ace921f17cfe4c81354a6c2fc5646e04575674a3e0e89f5f3a7b53c2b0b\": container with ID starting with c2f64ace921f17cfe4c81354a6c2fc5646e04575674a3e0e89f5f3a7b53c2b0b not found: ID does not exist" containerID="c2f64ace921f17cfe4c81354a6c2fc5646e04575674a3e0e89f5f3a7b53c2b0b"
Sep 12 01:13:43.656156 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:43.656114    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2f64ace921f17cfe4c81354a6c2fc5646e04575674a3e0e89f5f3a7b53c2b0b"} err="failed to get container status \"c2f64ace921f17cfe4c81354a6c2fc5646e04575674a3e0e89f5f3a7b53c2b0b\": rpc error: code = NotFound desc = could not find container \"c2f64ace921f17cfe4c81354a6c2fc5646e04575674a3e0e89f5f3a7b53c2b0b\": container with ID starting with c2f64ace921f17cfe4c81354a6c2fc5646e04575674a3e0e89f5f3a7b53c2b0b not found: ID does not exist"
Sep 12 01:13:43.667751 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:43.667730    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-84484d59ff-fqvpv"]
Sep 12 01:13:43.671912 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:43.671886    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-84484d59ff-fqvpv"]
Sep 12 01:13:43.738226 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:43.738196    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cf21cd5-aa7b-47a8-b2da-137f8b53f373" path="/var/lib/kubelet/pods/0cf21cd5-aa7b-47a8-b2da-137f8b53f373/volumes"
Sep 12 01:13:45.891610 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:45.891563    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-84484d59ff-fqvpv" podUID="0cf21cd5-aa7b-47a8-b2da-137f8b53f373" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.30:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Sep 12 01:13:53.090357 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:13:53.090318    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:14:01.913028 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:01.912985    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6cf675f9b4-jr6fr_342bbca2-a420-461f-be92-fa68b01c0f31/manager/0.log"
Sep 12 01:14:02.477367 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:02.477340    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67b7744c54-rgj84_9bc3e216-bd52-4f60-9ffa-b851f019174f/copy-manifests/0.log"
Sep 12 01:14:02.484128 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:02.484101    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67b7744c54-rgj84_9bc3e216-bd52-4f60-9ffa-b851f019174f/manager/0.log"
Sep 12 01:14:02.919179 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:02.919065    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-65b566db79-tjbz4_011c92b3-f7c0-46c5-bceb-ddbb69a30c13/maas-api/0.log"
Sep 12 01:14:03.238384 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:03.238350    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-5l4sh_c9c1d977-2a40-4397-bda4-813db9549e9f/postgres/0.log"
Sep 12 01:14:05.364426 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:05.364396    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-797549bd89-h8pbs_ef6df39b-1ed0-4873-a049-2bc77833c2a3/kube-auth-proxy/0.log"
Sep 12 01:14:05.734494 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:05.734468    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-lxcxb_0c452359-d6ce-4238-9ccd-73d99a638f4d/payload-pre-processing/0.log"
Sep 12 01:14:05.965474 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:05.965445    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-d6d5fbc68-m67sb_c47d77e0-78de-4d81-9e37-c9f5572e23a4/router/0.log"
Sep 12 01:14:06.435078 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:06.435047    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-lj2gn_a52bf419-7387-4576-914f-f8696242137a/main/0.log"
Sep 12 01:14:06.650747 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:06.650694    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-lrws2_0d4a6f94-092b-4fa0-be27-3748a7fe1658/main/0.log"
Sep 12 01:14:06.755193 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:06.755108    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f94j9n_c7aa001a-6286-432c-a666-682a4f925a2e/main/0.log"
Sep 12 01:14:12.973053 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:12.973018    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:14:13.109724 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:13.109686    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log" limit=1024
Sep 12 01:14:16.173436 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:16.173351    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-b2hjw_ed1e49a7-a9ae-4e55-8a01-c88326238a56/global-pull-secret-syncer/0.log"
Sep 12 01:14:16.323766 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:16.323734    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9dh42_cc3d5554-d660-453c-bc3e-4d3ed765400e/konnectivity-agent/0.log"
Sep 12 01:14:16.553437 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:16.553409    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-85.ec2.internal_83844964b8d09ab82b7ac30298c1090d/haproxy/0.log"
Sep 12 01:14:23.812790 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:23.812754    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-86b96456cd-zvh4c_da806a2e-7fdf-4973-b36b-b646682281a4/metrics-server/0.log"
Sep 12 01:14:23.901847 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:23.901807    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6mxtl_1d102e18-0228-4bb0-bc60-6c2942d7b321/node-exporter/0.log"
Sep 12 01:14:23.958549 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:23.958524    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-czf8h_dd90996a-5428-4ccf-a154-c0aa8def9789/console-operator/1.log"
Sep 12 01:14:23.971638 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:23.971616    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-czf8h_dd90996a-5428-4ccf-a154-c0aa8def9789/console-operator/1.log"
Sep 12 01:14:23.992125 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:23.992004    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6mxtl_1d102e18-0228-4bb0-bc60-6c2942d7b321/kube-rbac-proxy/0.log"
Sep 12 01:14:24.176099 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:24.176068    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6mxtl_1d102e18-0228-4bb0-bc60-6c2942d7b321/init-textfile/0.log"
Sep 12 01:14:24.703450 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:24.703421    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9b09956e-511a-40ce-94b5-a00ab565695d/prometheus/0.log"
Sep 12 01:14:24.732501 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:24.732468    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9b09956e-511a-40ce-94b5-a00ab565695d/config-reloader/0.log"
Sep 12 01:14:24.766611 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:24.766580    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9b09956e-511a-40ce-94b5-a00ab565695d/thanos-sidecar/0.log"
Sep 12 01:14:24.799417 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:24.799371    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9b09956e-511a-40ce-94b5-a00ab565695d/kube-rbac-proxy-web/0.log"
Sep 12 01:14:24.828794 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:24.828746    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9b09956e-511a-40ce-94b5-a00ab565695d/kube-rbac-proxy/0.log"
Sep 12 01:14:24.851412 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:24.851382    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9b09956e-511a-40ce-94b5-a00ab565695d/kube-rbac-proxy-thanos/0.log"
Sep 12 01:14:24.873751 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:24.873724    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9b09956e-511a-40ce-94b5-a00ab565695d/init-config-reloader/0.log"
Sep 12 01:14:24.906117 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:24.906079    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-nvmpr_73009400-1d5f-4778-8a76-50b67d0860b4/prometheus-operator/0.log"
Sep 12 01:14:24.927980 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:24.927959    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-nvmpr_73009400-1d5f-4778-8a76-50b67d0860b4/kube-rbac-proxy/0.log"
Sep 12 01:14:25.079534 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:25.079439    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84d9f6d8bf-grjpd_5f89c9a5-1e8c-4e54-be92-c104eb70007f/thanos-query/0.log"
Sep 12 01:14:25.101709 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:25.101676    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84d9f6d8bf-grjpd_5f89c9a5-1e8c-4e54-be92-c104eb70007f/kube-rbac-proxy-web/0.log"
Sep 12 01:14:25.126529 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:25.126503    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84d9f6d8bf-grjpd_5f89c9a5-1e8c-4e54-be92-c104eb70007f/kube-rbac-proxy/0.log"
Sep 12 01:14:25.147360 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:25.147331    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84d9f6d8bf-grjpd_5f89c9a5-1e8c-4e54-be92-c104eb70007f/prom-label-proxy/0.log"
Sep 12 01:14:25.169949 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:25.169918    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84d9f6d8bf-grjpd_5f89c9a5-1e8c-4e54-be92-c104eb70007f/kube-rbac-proxy-rules/0.log"
Sep 12 01:14:25.193103 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:25.193048    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84d9f6d8bf-grjpd_5f89c9a5-1e8c-4e54-be92-c104eb70007f/kube-rbac-proxy-metrics/0.log"
Sep 12 01:14:26.097415 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:26.097379    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fp67s/perf-node-gather-daemonset-vqn7x"]
Sep 12 01:14:26.097803 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:26.097760    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cf21cd5-aa7b-47a8-b2da-137f8b53f373" containerName="maas-api"
Sep 12 01:14:26.097803 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:26.097771    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cf21cd5-aa7b-47a8-b2da-137f8b53f373" containerName="maas-api"
Sep 12 01:14:26.097882 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:26.097834    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cf21cd5-aa7b-47a8-b2da-137f8b53f373" containerName="maas-api"
Sep 12 01:14:26.102033 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:26.102010    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fp67s/perf-node-gather-daemonset-vqn7x"
Sep 12 01:14:26.104860 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:26.104843    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fp67s\"/\"kube-root-ca.crt\""
Sep 12 01:14:26.106024 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:26.105998    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fp67s\"/\"openshift-service-ca.crt\""
Sep 12 01:14:26.106024 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:26.106019    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fp67s\"/\"default-dockercfg-wb842\""
Sep 12 01:14:26.119960 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:26.119933    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fp67s/perf-node-gather-daemonset-vqn7x"]
Sep 12 01:14:26.224004 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:26.223964    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5d254c49-aa6f-4c7f-b862-6f260353b5ea-proc\") pod \"perf-node-gather-daemonset-vqn7x\" (UID: \"5d254c49-aa6f-4c7f-b862-6f260353b5ea\") " pod="openshift-must-gather-fp67s/perf-node-gather-daemonset-vqn7x"
Sep 12 01:14:26.224242 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:26.224016    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5d254c49-aa6f-4c7f-b862-6f260353b5ea-podres\") pod \"perf-node-gather-daemonset-vqn7x\" (UID: \"5d254c49-aa6f-4c7f-b862-6f260353b5ea\") " pod="openshift-must-gather-fp67s/perf-node-gather-daemonset-vqn7x"
Sep 12 01:14:26.224242 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:26.224128    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5d254c49-aa6f-4c7f-b862-6f260353b5ea-sys\") pod \"perf-node-gather-daemonset-vqn7x\" (UID: \"5d254c49-aa6f-4c7f-b862-6f260353b5ea\") " pod="openshift-must-gather-fp67s/perf-node-gather-daemonset-vqn7x"
Sep 12 01:14:26.224242 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:26.224189    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wls28\" (UniqueName: \"kubernetes.io/projected/5d254c49-aa6f-4c7f-b862-6f260353b5ea-kube-api-access-wls28\") pod \"perf-node-gather-daemonset-vqn7x\" (UID: \"5d254c49-aa6f-4c7f-b862-6f260353b5ea\") " pod="openshift-must-gather-fp67s/perf-node-gather-daemonset-vqn7x"
Sep 12 01:14:26.224375 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:26.224305    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5d254c49-aa6f-4c7f-b862-6f260353b5ea-lib-modules\") pod \"perf-node-gather-daemonset-vqn7x\" (UID: \"5d254c49-aa6f-4c7f-b862-6f260353b5ea\") " pod="openshift-must-gather-fp67s/perf-node-gather-daemonset-vqn7x"
Sep 12 01:14:26.325483 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:26.325429    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5d254c49-aa6f-4c7f-b862-6f260353b5ea-lib-modules\") pod \"perf-node-gather-daemonset-vqn7x\" (UID: \"5d254c49-aa6f-4c7f-b862-6f260353b5ea\") " pod="openshift-must-gather-fp67s/perf-node-gather-daemonset-vqn7x"
Sep 12 01:14:26.325644 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:26.325515    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5d254c49-aa6f-4c7f-b862-6f260353b5ea-proc\") pod \"perf-node-gather-daemonset-vqn7x\" (UID: \"5d254c49-aa6f-4c7f-b862-6f260353b5ea\") " pod="openshift-must-gather-fp67s/perf-node-gather-daemonset-vqn7x"
Sep 12 01:14:26.325644 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:26.325542    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5d254c49-aa6f-4c7f-b862-6f260353b5ea-podres\") pod \"perf-node-gather-daemonset-vqn7x\" (UID: \"5d254c49-aa6f-4c7f-b862-6f260353b5ea\") " pod="openshift-must-gather-fp67s/perf-node-gather-daemonset-vqn7x"
Sep 12 01:14:26.325644 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:26.325579    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5d254c49-aa6f-4c7f-b862-6f260353b5ea-sys\") pod \"perf-node-gather-daemonset-vqn7x\" (UID: \"5d254c49-aa6f-4c7f-b862-6f260353b5ea\") " pod="openshift-must-gather-fp67s/perf-node-gather-daemonset-vqn7x"
Sep 12 01:14:26.325644 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:26.325618    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5d254c49-aa6f-4c7f-b862-6f260353b5ea-lib-modules\") pod \"perf-node-gather-daemonset-vqn7x\" (UID: \"5d254c49-aa6f-4c7f-b862-6f260353b5ea\") " pod="openshift-must-gather-fp67s/perf-node-gather-daemonset-vqn7x"
Sep 12 01:14:26.325644 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:26.325636    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5d254c49-aa6f-4c7f-b862-6f260353b5ea-proc\") pod \"perf-node-gather-daemonset-vqn7x\" (UID: \"5d254c49-aa6f-4c7f-b862-6f260353b5ea\") " pod="openshift-must-gather-fp67s/perf-node-gather-daemonset-vqn7x"
Sep 12 01:14:26.325853 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:26.325636    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wls28\" (UniqueName: \"kubernetes.io/projected/5d254c49-aa6f-4c7f-b862-6f260353b5ea-kube-api-access-wls28\") pod \"perf-node-gather-daemonset-vqn7x\" (UID: \"5d254c49-aa6f-4c7f-b862-6f260353b5ea\") " pod="openshift-must-gather-fp67s/perf-node-gather-daemonset-vqn7x"
Sep 12 01:14:26.325853 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:26.325673    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5d254c49-aa6f-4c7f-b862-6f260353b5ea-sys\") pod \"perf-node-gather-daemonset-vqn7x\" (UID: \"5d254c49-aa6f-4c7f-b862-6f260353b5ea\") " pod="openshift-must-gather-fp67s/perf-node-gather-daemonset-vqn7x"
Sep 12 01:14:26.325853 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:26.325703    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5d254c49-aa6f-4c7f-b862-6f260353b5ea-podres\") pod \"perf-node-gather-daemonset-vqn7x\" (UID: \"5d254c49-aa6f-4c7f-b862-6f260353b5ea\") " pod="openshift-must-gather-fp67s/perf-node-gather-daemonset-vqn7x"
Sep 12 01:14:26.335317 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:26.335292    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wls28\" (UniqueName: \"kubernetes.io/projected/5d254c49-aa6f-4c7f-b862-6f260353b5ea-kube-api-access-wls28\") pod \"perf-node-gather-daemonset-vqn7x\" (UID: \"5d254c49-aa6f-4c7f-b862-6f260353b5ea\") " pod="openshift-must-gather-fp67s/perf-node-gather-daemonset-vqn7x"
Sep 12 01:14:26.412225 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:26.412115    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fp67s/perf-node-gather-daemonset-vqn7x"
Sep 12 01:14:26.699924 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:26.699897    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fp67s/perf-node-gather-daemonset-vqn7x"]
Sep 12 01:14:26.702451 ip-10-0-142-85 kubenswrapper[2558]: W0912 01:14:26.702428    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5d254c49_aa6f_4c7f_b862_6f260353b5ea.slice/crio-2620ead59731a0c5f88b467736e38a932c8cb3137e3e25b9e9950550f3567fb3 WatchSource:0}: Error finding container 2620ead59731a0c5f88b467736e38a932c8cb3137e3e25b9e9950550f3567fb3: Status 404 returned error can't find the container with id 2620ead59731a0c5f88b467736e38a932c8cb3137e3e25b9e9950550f3567fb3
Sep 12 01:14:26.859599 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:26.859546    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fp67s/perf-node-gather-daemonset-vqn7x" event={"ID":"5d254c49-aa6f-4c7f-b862-6f260353b5ea","Type":"ContainerStarted","Data":"c98dfb3ead695dbda54043355bea452392c12e3e2c8c7106f127b12d05c5d26e"}
Sep 12 01:14:26.859778 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:26.859620    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fp67s/perf-node-gather-daemonset-vqn7x" event={"ID":"5d254c49-aa6f-4c7f-b862-6f260353b5ea","Type":"ContainerStarted","Data":"2620ead59731a0c5f88b467736e38a932c8cb3137e3e25b9e9950550f3567fb3"}
Sep 12 01:14:26.859778 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:26.859697    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-fp67s/perf-node-gather-daemonset-vqn7x"
Sep 12 01:14:27.369074 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:27.369045    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-czf8h_dd90996a-5428-4ccf-a154-c0aa8def9789/console-operator/1.log"
Sep 12 01:14:27.372832 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:27.372812    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-czf8h_dd90996a-5428-4ccf-a154-c0aa8def9789/console-operator/2.log"
Sep 12 01:14:28.047279 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:28.047247    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-55c4s_df843273-bc32-44d4-95bd-0ed4b1db206c/download-server/0.log"
Sep 12 01:14:29.929409 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:29.929382    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5wzpq_f0b419ed-e351-402a-9b9b-ff1534048f68/dns/0.log"
Sep 12 01:14:29.951275 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:29.951248    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5wzpq_f0b419ed-e351-402a-9b9b-ff1534048f68/kube-rbac-proxy/0.log"
Sep 12 01:14:30.091285 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:30.091250    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gs67t_34aa479e-166d-4d92-8909-a58f6e2649ef/dns-node-resolver/0.log"
Sep 12 01:14:30.874440 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:30.874412    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fd27q_c17577fa-fd1b-45a2-9d4d-6be7144ba759/node-ca/0.log"
Sep 12 01:14:32.492426 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:32.492391    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-797549bd89-h8pbs_ef6df39b-1ed0-4873-a049-2bc77833c2a3/kube-auth-proxy/0.log"
Sep 12 01:14:32.680159 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:32.680119    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-lxcxb_0c452359-d6ce-4238-9ccd-73d99a638f4d/payload-pre-processing/0.log"
Sep 12 01:14:32.811560 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:32.811457    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-d6d5fbc68-m67sb_c47d77e0-78de-4d81-9e37-c9f5572e23a4/router/0.log"
Sep 12 01:14:32.875039 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:32.875013    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-fp67s/perf-node-gather-daemonset-vqn7x"
Sep 12 01:14:32.902654 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:32.902607    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fp67s/perf-node-gather-daemonset-vqn7x" podStartSLOduration=6.902591848 podStartE2EDuration="6.902591848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-12 01:14:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-12 01:14:26.89033875 +0000 UTC m=+2703.564397608" watchObservedRunningTime="2026-09-12 01:14:32.902591848 +0000 UTC m=+2709.576650707"
Sep 12 01:14:33.917433 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:33.917405    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vcclb_421f63c6-8abc-4091-a546-2754a4f2066f/serve-healthcheck-canary/0.log"
Sep 12 01:14:34.475906 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:34.475875    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-cnml8_fc47c0e2-02fe-408b-b0a5-5b046a81f1d7/insights-operator/1.log"
Sep 12 01:14:34.476188 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:34.476010    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-cnml8_fc47c0e2-02fe-408b-b0a5-5b046a81f1d7/insights-operator/0.log"
Sep 12 01:14:34.857430 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:34.857355    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s4wrn_97666338-a119-4951-baeb-f794d6e6ac0b/kube-rbac-proxy/0.log"
Sep 12 01:14:34.942239 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:34.942202    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s4wrn_97666338-a119-4951-baeb-f794d6e6ac0b/exporter/0.log"
Sep 12 01:14:35.002506 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:35.002463    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s4wrn_97666338-a119-4951-baeb-f794d6e6ac0b/extractor/0.log"
Sep 12 01:14:37.503668 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:37.503636    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-797549bd89-h8pbs_ef6df39b-1ed0-4873-a049-2bc77833c2a3/kube-auth-proxy/0.log"
Sep 12 01:14:37.598439 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:37.598358    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-lxcxb_0c452359-d6ce-4238-9ccd-73d99a638f4d/payload-pre-processing/0.log"
Sep 12 01:14:37.660694 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:37.660666    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-d6d5fbc68-m67sb_c47d77e0-78de-4d81-9e37-c9f5572e23a4/router/0.log"
Sep 12 01:14:38.121496 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:38.121468    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6cf675f9b4-jr6fr_342bbca2-a420-461f-be92-fa68b01c0f31/manager/0.log"
Sep 12 01:14:38.559964 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:38.559934    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67b7744c54-rgj84_9bc3e216-bd52-4f60-9ffa-b851f019174f/manager/0.log"
Sep 12 01:14:38.581213 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:38.581184    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67b7744c54-rgj84_9bc3e216-bd52-4f60-9ffa-b851f019174f/copy-manifests/0.log"
Sep 12 01:14:39.815322 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:39.815291    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-797549bd89-h8pbs_ef6df39b-1ed0-4873-a049-2bc77833c2a3/kube-auth-proxy/0.log"
Sep 12 01:14:39.907348 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:39.907318    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7bc7647c9b-82h5h_030ba73a-88ac-488a-ac3e-7b12fd28bf27/manager/0.log"
Sep 12 01:14:39.926758 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:39.926730    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-lxcxb_0c452359-d6ce-4238-9ccd-73d99a638f4d/payload-pre-processing/0.log"
Sep 12 01:14:39.949038 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:39.949013    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-jgslq_6686840f-1e9a-4fdf-87d0-0c7222538f54/openshift-lws-operator/0.log"
Sep 12 01:14:39.995639 ip-10-0-142-85 kubenswrapper[2558]: I0912 01:14:39.995615    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-d6d5fbc68-m67sb_c47d77e0-78de-4d81-9e37-c9f5572e23a4/router/0.log"