Sep 11 00:56:03.596583 ip-10-0-139-235 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 11 00:56:03.596599 ip-10-0-139-235 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 11 00:56:03.596609 ip-10-0-139-235 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 11 00:56:03.596949 ip-10-0-139-235 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 11 00:56:13.696745 ip-10-0-139-235 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 11 00:56:13.696764 ip-10-0-139-235 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 8f5f96cff72748b6a4428ce349d0f41d --
Sep 11 00:58:04.482197 ip-10-0-139-235 systemd[1]: Starting Kubernetes Kubelet...
Sep 11 00:58:04.900982 ip-10-0-139-235 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 00:58:04.900982 ip-10-0-139-235 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 11 00:58:04.900982 ip-10-0-139-235 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 00:58:04.900982 ip-10-0-139-235 kubenswrapper[2574]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 11 00:58:04.900982 ip-10-0-139-235 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 00:58:04.901951 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.901858    2574 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 11 00:58:04.906019 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906004    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 00:58:04.906019 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906019    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 00:58:04.906077 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906023    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 00:58:04.906077 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906026    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 00:58:04.906077 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906028    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 00:58:04.906077 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906031    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 00:58:04.906077 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906034    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 00:58:04.906077 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906037    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 00:58:04.906077 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906040    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 00:58:04.906077 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906042    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 00:58:04.906077 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906045    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 00:58:04.906077 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906047    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 00:58:04.906077 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906050    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 00:58:04.906077 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906052    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 00:58:04.906077 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906055    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 00:58:04.906077 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906057    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 00:58:04.906077 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906060    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 00:58:04.906077 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906062    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 00:58:04.906077 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906066    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 00:58:04.906077 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906070    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 00:58:04.906077 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906074    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 00:58:04.906540 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906077    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 00:58:04.906540 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906080    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 00:58:04.906540 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906083    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 00:58:04.906540 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906086    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 00:58:04.906540 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906089    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 00:58:04.906540 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906092    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 00:58:04.906540 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906094    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 00:58:04.906540 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906097    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 00:58:04.906540 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906099    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 00:58:04.906540 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906102    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 00:58:04.906540 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906104    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 00:58:04.906540 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906107    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 00:58:04.906540 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906109    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 00:58:04.906540 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906112    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 00:58:04.906540 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906114    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 00:58:04.906540 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906117    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 00:58:04.906540 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906119    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 00:58:04.906540 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906122    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 00:58:04.906540 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906124    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 00:58:04.906540 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906128    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 00:58:04.907063 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906130    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 00:58:04.907063 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906133    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 00:58:04.907063 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906135    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 00:58:04.907063 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906138    2574 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 00:58:04.907063 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906140    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 00:58:04.907063 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906143    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 00:58:04.907063 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906145    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 00:58:04.907063 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906147    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 00:58:04.907063 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906151    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 00:58:04.907063 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906153    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 00:58:04.907063 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906156    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 00:58:04.907063 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906158    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 00:58:04.907063 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906160    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 00:58:04.907063 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906164    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 00:58:04.907063 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906167    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 00:58:04.907063 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906171    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 00:58:04.907063 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906174    2574 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 00:58:04.907063 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906178    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 00:58:04.907063 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906181    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 00:58:04.907523 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906184    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 00:58:04.907523 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906187    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 00:58:04.907523 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906189    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 00:58:04.907523 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906192    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 00:58:04.907523 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906194    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 00:58:04.907523 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906197    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 00:58:04.907523 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906199    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 00:58:04.907523 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906202    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 00:58:04.907523 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906204    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 00:58:04.907523 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906207    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 00:58:04.907523 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906209    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 00:58:04.907523 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906212    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 00:58:04.907523 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906214    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 00:58:04.907523 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906217    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 00:58:04.907523 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906220    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 00:58:04.907523 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906222    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 00:58:04.907523 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906224    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 00:58:04.907523 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906227    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 00:58:04.907523 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906230    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 00:58:04.907523 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906232    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 00:58:04.908011 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906234    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 00:58:04.908011 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906238    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 00:58:04.908011 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906240    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 00:58:04.908011 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906243    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 00:58:04.908011 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906245    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 00:58:04.908011 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906248    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 00:58:04.908011 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906251    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 00:58:04.908011 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906253    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 00:58:04.908011 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906258    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 00:58:04.908011 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906261    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 00:58:04.908011 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906264    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 00:58:04.908011 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906267    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 00:58:04.908011 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906270    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 00:58:04.908011 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906273    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 00:58:04.908011 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906276    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 00:58:04.908011 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906278    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 00:58:04.908011 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906281    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 00:58:04.908011 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906283    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 00:58:04.908011 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906286    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 00:58:04.908497 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906986    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 00:58:04.908497 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906993    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 00:58:04.908497 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906996    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 00:58:04.908497 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.906999    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 00:58:04.908497 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907001    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 00:58:04.908497 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907004    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 00:58:04.908497 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907007    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 00:58:04.908497 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907010    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 00:58:04.908497 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907013    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 00:58:04.908497 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907015    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 00:58:04.908497 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907019    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 00:58:04.908497 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907021    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 00:58:04.908497 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907024    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 00:58:04.908497 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907027    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 00:58:04.908497 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907029    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 00:58:04.908497 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907032    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 00:58:04.908497 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907034    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 00:58:04.908497 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907037    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 00:58:04.908497 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907040    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 00:58:04.908497 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907043    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 00:58:04.909296 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907045    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 00:58:04.909296 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907048    2574 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 00:58:04.909296 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907050    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 00:58:04.909296 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907053    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 00:58:04.909296 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907056    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 00:58:04.909296 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907058    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 00:58:04.909296 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907061    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 00:58:04.909296 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907063    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 00:58:04.909296 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907065    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 00:58:04.909296 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907068    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 00:58:04.909296 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907071    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 00:58:04.909296 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907073    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 00:58:04.909296 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907076    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 00:58:04.909296 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907079    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 00:58:04.909296 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907081    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 00:58:04.909296 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907084    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 00:58:04.909296 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907086    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 00:58:04.909296 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907089    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 00:58:04.909296 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907092    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 00:58:04.909296 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907094    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 00:58:04.909817 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907097    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 00:58:04.909817 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907100    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 00:58:04.909817 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907103    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 00:58:04.909817 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907106    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 00:58:04.909817 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907110    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 00:58:04.909817 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907113    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 00:58:04.909817 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907115    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 00:58:04.909817 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907118    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 00:58:04.909817 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907120    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 00:58:04.909817 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907122    2574 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 00:58:04.909817 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907125    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 00:58:04.909817 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907128    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 00:58:04.909817 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907130    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 00:58:04.909817 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907135    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 00:58:04.909817 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907138    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 00:58:04.909817 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907142    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 00:58:04.909817 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907145    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 00:58:04.909817 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907148    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 00:58:04.909817 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907150    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 00:58:04.910301 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907153    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 00:58:04.910301 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907156    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 00:58:04.910301 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907158    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 00:58:04.910301 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907161    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 00:58:04.910301 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907164    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 00:58:04.910301 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907166    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 00:58:04.910301 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907169    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 00:58:04.910301 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907171    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 00:58:04.910301 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907174    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 00:58:04.910301 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907176    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 00:58:04.910301 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907179    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 00:58:04.910301 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907181    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 00:58:04.910301 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907184    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 00:58:04.910301 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907186    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 00:58:04.910301 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907189    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 00:58:04.910301 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907192    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 00:58:04.910301 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907194    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 00:58:04.910301 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907197    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 00:58:04.910301 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907200    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 00:58:04.910301 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907202    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 00:58:04.910786 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907205    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 00:58:04.910786 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907207    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 00:58:04.910786 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907210    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 00:58:04.910786 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907212    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 00:58:04.910786 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907215    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 00:58:04.910786 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907218    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 00:58:04.910786 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907220    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 00:58:04.910786 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907224    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 00:58:04.910786 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907226    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 00:58:04.910786 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907229    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 00:58:04.910786 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907231    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 00:58:04.910786 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907234    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 00:58:04.910786 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907236    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 00:58:04.910786 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907238    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 00:58:04.910786 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907241    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 00:58:04.910786 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907244    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 00:58:04.910786 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907248    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 00:58:04.910786 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907251    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 00:58:04.910786 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907254    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 00:58:04.911355 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907257    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 00:58:04.911355 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907589    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 00:58:04.911355 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907593    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 00:58:04.911355 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907596    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 00:58:04.911355 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907600    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 00:58:04.911355 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907603    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 00:58:04.911355 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907605    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 00:58:04.911355 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907609    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 00:58:04.911355 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907612    2574 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 00:58:04.911355 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907615    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 00:58:04.911355 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907618    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 00:58:04.911355 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907620    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 00:58:04.911355 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907623    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 00:58:04.911355 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907625    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 00:58:04.911355 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907628    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 00:58:04.911355 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907631    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 00:58:04.911355 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907634    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 00:58:04.911355 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907636    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 00:58:04.911355 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907639    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 00:58:04.911355 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907641    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 00:58:04.911832 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907644    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 00:58:04.911832 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907646    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 00:58:04.911832 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907649    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 00:58:04.911832 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907651    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 00:58:04.911832 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907654    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 00:58:04.911832 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907656    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 00:58:04.911832 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907659    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 00:58:04.911832 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907663    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 00:58:04.911832 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907666    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 00:58:04.911832 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907669    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 00:58:04.911832 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907672    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 00:58:04.911832 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907675    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 00:58:04.911832 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907678    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 00:58:04.911832 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907680    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 00:58:04.911832 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907683    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 00:58:04.911832 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907686    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 00:58:04.911832 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907689    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 00:58:04.911832 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907692    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 00:58:04.911832 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907695    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 00:58:04.912318 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907697    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 00:58:04.912318 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907701    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 00:58:04.912318 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907703    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 00:58:04.912318 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907706    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 00:58:04.912318 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907708    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 00:58:04.912318 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907711    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 00:58:04.912318 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907713    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 00:58:04.912318 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907716    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 00:58:04.912318 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907718    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 00:58:04.912318 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907722    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 00:58:04.912318 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907724    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 00:58:04.912318 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907726    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 00:58:04.912318 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907729    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 00:58:04.912318 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907733    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 00:58:04.912318 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907736    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 00:58:04.912318 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907738    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 00:58:04.912318 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907741    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 00:58:04.912318 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907743    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 00:58:04.912318 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907746    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 00:58:04.912785 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907748    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 00:58:04.912785 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907751    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 00:58:04.912785 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907753    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 00:58:04.912785 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907756    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 00:58:04.912785 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907758    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 00:58:04.912785 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907760    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 00:58:04.912785 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907763    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 00:58:04.912785 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907765    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 00:58:04.912785 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907768    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 00:58:04.912785 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907771    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 00:58:04.912785 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907773    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 00:58:04.912785 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907776    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 00:58:04.912785 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907778    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 00:58:04.912785 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907781    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 00:58:04.912785 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907785    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 00:58:04.912785 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907788    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 00:58:04.912785 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907791    2574 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 00:58:04.912785 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907794    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 00:58:04.912785 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907796    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 00:58:04.912785 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907800    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 00:58:04.913288 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907804    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 00:58:04.913288 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907807    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 00:58:04.913288 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907810    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 00:58:04.913288 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907813    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 00:58:04.913288 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907815    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 00:58:04.913288 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907817    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 00:58:04.913288 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907820    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 00:58:04.913288 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907822    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 00:58:04.913288 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907825    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 00:58:04.913288 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907827    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 00:58:04.913288 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907830    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 00:58:04.913288 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907832    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 00:58:04.913288 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907834    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 00:58:04.913288 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907837    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 00:58:04.913288 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907840    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 00:58:04.913288 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907842    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 00:58:04.913288 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907845    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 00:58:04.913288 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907847    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 00:58:04.913288 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907849    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 00:58:04.913288 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907852    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 00:58:04.913775 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907854    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 00:58:04.913775 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.907857    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 00:58:04.913775 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.907944    2574 flags.go:64] FLAG: --address="0.0.0.0"
Sep 11 00:58:04.913775 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.907984    2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 11 00:58:04.913775 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.907990    2574 flags.go:64] FLAG: --anonymous-auth="true"
Sep 11 00:58:04.913775 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.907994    2574 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 11 00:58:04.913775 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.907999    2574 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 11 00:58:04.913775 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908002    2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 11 00:58:04.913775 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908006    2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 11 00:58:04.913775 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908010    2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 11 00:58:04.913775 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908014    2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 11 00:58:04.913775 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908017    2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 11 00:58:04.913775 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908020    2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 11 00:58:04.913775 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908023    2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 11 00:58:04.913775 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908026    2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 11 00:58:04.913775 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908029    2574 flags.go:64] FLAG: --cgroup-root=""
Sep 11 00:58:04.913775 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908031    2574 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 11 00:58:04.913775 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908034    2574 flags.go:64] FLAG: --client-ca-file=""
Sep 11 00:58:04.913775 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908037    2574 flags.go:64] FLAG: --cloud-provider="external"
Sep 11 00:58:04.913775 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908040    2574 flags.go:64] FLAG: --cluster-dns="[]"
Sep 11 00:58:04.913775 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908044    2574 flags.go:64] FLAG: --cluster-domain=""
Sep 11 00:58:04.913775 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908046    2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 11 00:58:04.913775 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908049    2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 11 00:58:04.913775 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908052    2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 11 00:58:04.914341 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908055    2574 flags.go:64] FLAG: --container-log-max-files="5"
Sep 11 00:58:04.914341 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908059    2574 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 11 00:58:04.914341 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908062    2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 11 00:58:04.914341 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908065    2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 11 00:58:04.914341 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908068    2574 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 11 00:58:04.914341 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908070    2574 flags.go:64] FLAG: --contention-profiling="false"
Sep 11 00:58:04.914341 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908073    2574 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 11 00:58:04.914341 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908076    2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 11 00:58:04.914341 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908079    2574 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 11 00:58:04.914341 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908082    2574 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 11 00:58:04.914341 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908087    2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 11 00:58:04.914341 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908090    2574 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 11 00:58:04.914341 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908093    2574 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 11 00:58:04.914341 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908095    2574 flags.go:64] FLAG: --enable-load-reader="false"
Sep 11 00:58:04.914341 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908101    2574 flags.go:64] FLAG: --enable-server="true"
Sep 11 00:58:04.914341 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908104    2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 11 00:58:04.914341 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908108    2574 flags.go:64] FLAG: --event-burst="100"
Sep 11 00:58:04.914341 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908111    2574 flags.go:64] FLAG: --event-qps="50"
Sep 11 00:58:04.914341 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908114    2574 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 11 00:58:04.914341 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908117    2574 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 11 00:58:04.914341 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908120    2574 flags.go:64] FLAG: --eviction-hard=""
Sep 11 00:58:04.914341 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908123    2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 11 00:58:04.914341 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908126    2574 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 11 00:58:04.914341 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908129    2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 11 00:58:04.914341 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908132    2574 flags.go:64] FLAG: --eviction-soft=""
Sep 11 00:58:04.914968 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908135    2574 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 11 00:58:04.914968 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908137    2574 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 11 00:58:04.914968 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908140    2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 11 00:58:04.914968 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908143    2574 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 11 00:58:04.914968 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908146    2574 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 11 00:58:04.914968 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908148    2574 flags.go:64] FLAG: --fail-swap-on="true"
Sep 11 00:58:04.914968 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908151    2574 flags.go:64] FLAG: --feature-gates=""
Sep 11 00:58:04.914968 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908154    2574 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 11 00:58:04.914968 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908157    2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 11 00:58:04.914968 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908160    2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 11 00:58:04.914968 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908163    2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 11 00:58:04.914968 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908166    2574 flags.go:64] FLAG: --healthz-port="10248"
Sep 11 00:58:04.914968 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908169    2574 flags.go:64] FLAG: --help="false"
Sep 11 00:58:04.914968 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908171    2574 flags.go:64] FLAG: --hostname-override="ip-10-0-139-235.ec2.internal"
Sep 11 00:58:04.914968 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908174    2574 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 11 00:58:04.914968 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908177    2574 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 11 00:58:04.914968 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908180    2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 11 00:58:04.914968 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908184    2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 11 00:58:04.914968 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908187    2574 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 11 00:58:04.914968 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908190    2574 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 11 00:58:04.914968 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908193    2574 flags.go:64] FLAG: --image-service-endpoint=""
Sep 11 00:58:04.914968 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908197    2574 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 11 00:58:04.914968 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908199    2574 flags.go:64] FLAG: --kube-api-burst="100"
Sep 11 00:58:04.915559 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908203    2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 11 00:58:04.915559 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908205    2574 flags.go:64] FLAG: --kube-api-qps="50"
Sep 11 00:58:04.915559 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908211    2574 flags.go:64] FLAG: --kube-reserved=""
Sep 11 00:58:04.915559 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908214    2574 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 11 00:58:04.915559 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908217    2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 11 00:58:04.915559 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908220    2574 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 11 00:58:04.915559 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908223    2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 11 00:58:04.915559 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908225    2574 flags.go:64] FLAG: --lock-file=""
Sep 11 00:58:04.915559 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908228    2574 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 11 00:58:04.915559 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908230    2574 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 11 00:58:04.915559 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908233    2574 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 11 00:58:04.915559 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908238    2574 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 11 00:58:04.915559 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908241    2574 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 11 00:58:04.915559 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908244    2574 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 11 00:58:04.915559 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908246    2574 flags.go:64] FLAG: --logging-format="text"
Sep 11 00:58:04.915559 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908249    2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 11 00:58:04.915559 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908252    2574 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 11 00:58:04.915559 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908255    2574 flags.go:64] FLAG: --manifest-url=""
Sep 11 00:58:04.915559 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908258    2574 flags.go:64] FLAG: --manifest-url-header=""
Sep 11 00:58:04.915559 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908262    2574 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 11 00:58:04.915559 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908265    2574 flags.go:64] FLAG: --max-open-files="1000000"
Sep 11 00:58:04.915559 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908268    2574 flags.go:64] FLAG: --max-pods="110"
Sep 11 00:58:04.915559 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908271    2574 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 11 00:58:04.915559 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908274    2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 11 00:58:04.915559 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908276    2574 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 11 00:58:04.916209 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908280    2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 11 00:58:04.916209 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908284    2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 11 00:58:04.916209 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908286    2574 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 11 00:58:04.916209 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908289    2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 11 00:58:04.916209 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908296    2574 flags.go:64] FLAG: --node-status-max-images="50"
Sep 11 00:58:04.916209 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908300    2574 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 11 00:58:04.916209 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908303    2574 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 11 00:58:04.916209 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908306    2574 flags.go:64] FLAG: --pod-cidr=""
Sep 11 00:58:04.916209 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908309    2574 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 11 00:58:04.916209 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908313    2574 flags.go:64] FLAG: --pod-manifest-path=""
Sep 11 00:58:04.916209 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908317    2574 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 11 00:58:04.916209 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908320    2574 flags.go:64] FLAG: --pods-per-core="0"
Sep 11 00:58:04.916209 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908323    2574 flags.go:64] FLAG: --port="10250"
Sep 11 00:58:04.916209 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908326    2574 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 11 00:58:04.916209 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908328    2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-059b60b8ff87166d2"
Sep 11 00:58:04.916209 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908331    2574 flags.go:64] FLAG: --qos-reserved=""
Sep 11 00:58:04.916209 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908334    2574 flags.go:64] FLAG: --read-only-port="10255"
Sep 11 00:58:04.916209 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908337    2574 flags.go:64] FLAG: --register-node="true"
Sep 11 00:58:04.916209 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908339    2574 flags.go:64] FLAG: --register-with-taints=""
Sep 11 00:58:04.916209 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908343    2574 flags.go:64] FLAG: --registry-burst="10"
Sep 11 00:58:04.916209 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908346    2574 flags.go:64] FLAG: --registry-qps="5"
Sep 11 00:58:04.916209 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908348    2574 flags.go:64] FLAG: --reserved-cpus=""
Sep 11 00:58:04.916209 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908351    2574 flags.go:64] FLAG: --reserved-memory=""
Sep 11 00:58:04.916209 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908354    2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 11 00:58:04.916209 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908357    2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 11 00:58:04.916808 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908360    2574 flags.go:64] FLAG: --rotate-certificates="false"
Sep 11 00:58:04.916808 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908363    2574 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 11 00:58:04.916808 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908366    2574 flags.go:64] FLAG: --runonce="false"
Sep 11 00:58:04.916808 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908369    2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 11 00:58:04.916808 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908371    2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 11 00:58:04.916808 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908374    2574 flags.go:64] FLAG: --seccomp-default="false"
Sep 11 00:58:04.916808 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908377    2574 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 11 00:58:04.916808 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908380    2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 11 00:58:04.916808 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908384    2574 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 11 00:58:04.916808 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908387    2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 11 00:58:04.916808 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908390    2574 flags.go:64] FLAG: --storage-driver-password="root"
Sep 11 00:58:04.916808 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908392    2574 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 11 00:58:04.916808 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908395    2574 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 11 00:58:04.916808 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908399    2574 flags.go:64] FLAG: --storage-driver-user="root"
Sep 11 00:58:04.916808 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908401    2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 11 00:58:04.916808 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908404    2574 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 11 00:58:04.916808 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908407    2574 flags.go:64] FLAG: --system-cgroups=""
Sep 11 00:58:04.916808 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908410    2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 11 00:58:04.916808 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908415    2574 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 11 00:58:04.916808 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908418    2574 flags.go:64] FLAG: --tls-cert-file=""
Sep 11 00:58:04.916808 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908421    2574 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 11 00:58:04.916808 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908425    2574 flags.go:64] FLAG: --tls-min-version=""
Sep 11 00:58:04.916808 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908427    2574 flags.go:64] FLAG: --tls-private-key-file=""
Sep 11 00:58:04.916808 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908430    2574 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 11 00:58:04.916808 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908433    2574 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 11 00:58:04.917424 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908436    2574 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 11 00:58:04.917424 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908439    2574 flags.go:64] FLAG: --v="2"
Sep 11 00:58:04.917424 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908443    2574 flags.go:64] FLAG: --version="false"
Sep 11 00:58:04.917424 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908446    2574 flags.go:64] FLAG: --vmodule=""
Sep 11 00:58:04.917424 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908450    2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 11 00:58:04.917424 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.908453    2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 11 00:58:04.917424 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909617    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 00:58:04.917424 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909630    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 00:58:04.917424 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909634    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 00:58:04.917424 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909637    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 00:58:04.917424 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909640    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 00:58:04.917424 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909644    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 00:58:04.917424 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909647    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 00:58:04.917424 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909654    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 00:58:04.917424 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909657    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 00:58:04.917424 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909660    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 00:58:04.917424 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909663    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 00:58:04.917424 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909666    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 00:58:04.917424 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909669    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 00:58:04.917424 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909672    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 00:58:04.917424 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909675    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 00:58:04.917958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909677    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 00:58:04.917958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909680    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 00:58:04.917958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909683    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 00:58:04.917958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909686    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 00:58:04.917958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909689    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 00:58:04.917958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909707    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 00:58:04.917958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909711    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 00:58:04.917958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909714    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 00:58:04.917958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909717    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 00:58:04.917958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909720    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 00:58:04.917958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909723    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 00:58:04.917958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909725    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 00:58:04.917958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909728    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 00:58:04.917958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909731    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 00:58:04.917958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909734    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 00:58:04.917958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909736    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 00:58:04.917958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909739    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 00:58:04.917958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909742    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 00:58:04.917958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909747    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 00:58:04.917958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909750    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 00:58:04.918478 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909753    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 00:58:04.918478 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909756    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 00:58:04.918478 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909759    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 00:58:04.918478 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909762    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 00:58:04.918478 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909766    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 00:58:04.918478 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909770    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 00:58:04.918478 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909773    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 00:58:04.918478 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909776    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 00:58:04.918478 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909794    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 00:58:04.918478 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909797    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 00:58:04.918478 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909800    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 00:58:04.918478 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909803    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 00:58:04.918478 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909805    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 00:58:04.918478 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909809    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 00:58:04.918478 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909812    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 00:58:04.918478 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909815    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 00:58:04.918478 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909818    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 00:58:04.918478 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909820    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 00:58:04.918478 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909850    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 00:58:04.918958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909964    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 00:58:04.918958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909970    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 00:58:04.918958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909974    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 00:58:04.918958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909980    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 00:58:04.918958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909984    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 00:58:04.918958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909988    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 00:58:04.918958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909990    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 00:58:04.918958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909993    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 00:58:04.918958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909996    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 00:58:04.918958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.909998    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 00:58:04.918958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910001    2574 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 00:58:04.918958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910004    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 00:58:04.918958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910007    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 00:58:04.918958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910009    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 00:58:04.918958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910012    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 00:58:04.918958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910015    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 00:58:04.918958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910017    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 00:58:04.918958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910019    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 00:58:04.918958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910022    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 00:58:04.918958 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910024    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 00:58:04.919438 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910027    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 00:58:04.919438 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910030    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 00:58:04.919438 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910032    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 00:58:04.919438 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910035    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 00:58:04.919438 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910037    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 00:58:04.919438 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910041    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 00:58:04.919438 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910045    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 00:58:04.919438 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910047    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 00:58:04.919438 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910051    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 00:58:04.919438 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910054    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 00:58:04.919438 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910057    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 00:58:04.919438 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910060    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 00:58:04.919438 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910064    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 00:58:04.919438 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910067    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 00:58:04.919438 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910070    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 00:58:04.919438 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910072    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 00:58:04.919438 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910075    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 00:58:04.919438 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910077    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 00:58:04.919438 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910080    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 00:58:04.919895 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910082    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 00:58:04.919895 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910085    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 00:58:04.919895 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910088    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 00:58:04.919895 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910090    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 00:58:04.919895 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910092    2574 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 00:58:04.919895 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.910095    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 00:58:04.919895 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.910742    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 00:58:04.919895 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.917414    2574 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 11 00:58:04.919895 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.917440    2574 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 11 00:58:04.919895 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.917459    2574 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 11 00:58:04.919895 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.917465    2574 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 11 00:58:04.919895 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917521    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 00:58:04.919895 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917527    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 00:58:04.919895 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917531    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 00:58:04.920259 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917535    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 00:58:04.920259 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917538    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 00:58:04.920259 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917541    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 00:58:04.920259 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917543    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 00:58:04.920259 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917546    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 00:58:04.920259 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917549    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 00:58:04.920259 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917551    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 00:58:04.920259 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917554    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 00:58:04.920259 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917557    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 00:58:04.920259 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917559    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 00:58:04.920259 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917562    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 00:58:04.920259 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917564    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 00:58:04.920259 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917567    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 00:58:04.920259 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917570    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 00:58:04.920259 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917572    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 00:58:04.920259 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917574    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 00:58:04.920259 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917577    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 00:58:04.920259 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917579    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 00:58:04.920259 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917582    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 00:58:04.920259 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917584    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 00:58:04.920744 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917587    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 00:58:04.920744 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917590    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 00:58:04.920744 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917592    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 00:58:04.920744 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917595    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 00:58:04.920744 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917597    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 00:58:04.920744 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917600    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 00:58:04.920744 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917603    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 00:58:04.920744 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917606    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 00:58:04.920744 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917608    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 00:58:04.920744 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917611    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 00:58:04.920744 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917615    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 00:58:04.920744 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917619    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 00:58:04.920744 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917622    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 00:58:04.920744 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917624    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 00:58:04.920744 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917627    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 00:58:04.920744 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917630    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 00:58:04.920744 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917632    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 00:58:04.920744 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917635    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 00:58:04.920744 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917638    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 00:58:04.921219 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917640    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 00:58:04.921219 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917643    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 00:58:04.921219 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917645    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 00:58:04.921219 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917647    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 00:58:04.921219 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917650    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 00:58:04.921219 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917652    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 00:58:04.921219 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917655    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 00:58:04.921219 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917657    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 00:58:04.921219 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917660    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 00:58:04.921219 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917663    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 00:58:04.921219 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917665    2574 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 00:58:04.921219 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917668    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 00:58:04.921219 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917671    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 00:58:04.921219 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917673    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 00:58:04.921219 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917676    2574 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 00:58:04.921219 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917678    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 00:58:04.921219 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917681    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 00:58:04.921219 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917683    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 00:58:04.921219 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917686    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 00:58:04.921219 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917689    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 00:58:04.921742 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917691    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 00:58:04.921742 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917695    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 00:58:04.921742 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917697    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 00:58:04.921742 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917700    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 00:58:04.921742 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917703    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 00:58:04.921742 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917705    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 00:58:04.921742 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917708    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 00:58:04.921742 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917711    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 00:58:04.921742 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917713    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 00:58:04.921742 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917716    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 00:58:04.921742 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917718    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 00:58:04.921742 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917720    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 00:58:04.921742 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917723    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 00:58:04.921742 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917725    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 00:58:04.921742 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917729    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 00:58:04.921742 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917733    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 00:58:04.921742 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917736    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 00:58:04.921742 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917739    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 00:58:04.921742 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917742    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 00:58:04.922213 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917744    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 00:58:04.922213 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917747    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 00:58:04.922213 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917749    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 00:58:04.922213 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917752    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 00:58:04.922213 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917754    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 00:58:04.922213 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917757    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 00:58:04.922213 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917759    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 00:58:04.922213 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917762    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 00:58:04.922213 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917764    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 00:58:04.922213 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917767    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 00:58:04.922213 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917769    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 00:58:04.922213 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917772    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 00:58:04.922213 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917775    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 00:58:04.922213 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917778    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 00:58:04.922213 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917781    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 00:58:04.922213 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917783    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 00:58:04.922213 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917786    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 00:58:04.922213 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917789    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 00:58:04.922650 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.917794    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 00:58:04.922650 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917927    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 00:58:04.922650 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917932    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 00:58:04.922650 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917935    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 00:58:04.922650 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917938    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 00:58:04.922650 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917941    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 00:58:04.922650 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917944    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 00:58:04.922650 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917947    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 00:58:04.922650 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917949    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 00:58:04.922650 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917952    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 00:58:04.922650 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917955    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 00:58:04.922650 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917957    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 00:58:04.922650 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917960    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 00:58:04.922650 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917963    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 00:58:04.922650 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917966    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 00:58:04.923031 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917968    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 00:58:04.923031 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917971    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 00:58:04.923031 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917974    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 00:58:04.923031 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917976    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 00:58:04.923031 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917979    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 00:58:04.923031 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917982    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 00:58:04.923031 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917984    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 00:58:04.923031 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917987    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 00:58:04.923031 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917989    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 00:58:04.923031 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917991    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 00:58:04.923031 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917994    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 00:58:04.923031 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.917997    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 00:58:04.923031 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918000    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 00:58:04.923031 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918003    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 00:58:04.923031 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918005    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 00:58:04.923031 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918008    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 00:58:04.923031 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918011    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 00:58:04.923031 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918013    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 00:58:04.923031 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918016    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 00:58:04.923031 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918019    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 00:58:04.923506 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918022    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 00:58:04.923506 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918024    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 00:58:04.923506 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918026    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 00:58:04.923506 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918029    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 00:58:04.923506 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918032    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 00:58:04.923506 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918034    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 00:58:04.923506 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918037    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 00:58:04.923506 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918039    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 00:58:04.923506 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918042    2574 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 00:58:04.923506 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918044    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 00:58:04.923506 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918046    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 00:58:04.923506 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918049    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 00:58:04.923506 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918052    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 00:58:04.923506 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918054    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 00:58:04.923506 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918057    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 00:58:04.923506 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918059    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 00:58:04.923506 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918062    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 00:58:04.923506 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918064    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 00:58:04.923506 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918066    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 00:58:04.924022 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918069    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 00:58:04.924022 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918071    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 00:58:04.924022 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918074    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 00:58:04.924022 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918076    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 00:58:04.924022 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918078    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 00:58:04.924022 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918081    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 00:58:04.924022 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918084    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 00:58:04.924022 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918086    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 00:58:04.924022 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918089    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 00:58:04.924022 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918091    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 00:58:04.924022 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918094    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 00:58:04.924022 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918096    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 00:58:04.924022 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918099    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 00:58:04.924022 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918101    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 00:58:04.924022 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918104    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 00:58:04.924022 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918106    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 00:58:04.924022 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918109    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 00:58:04.924022 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918111    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 00:58:04.924022 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918114    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 00:58:04.924022 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918117    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 00:58:04.924494 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918120    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 00:58:04.924494 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918124    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 00:58:04.924494 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918128    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 00:58:04.924494 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918131    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 00:58:04.924494 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918134    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 00:58:04.924494 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918137    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 00:58:04.924494 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918140    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 00:58:04.924494 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918142    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 00:58:04.924494 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918145    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 00:58:04.924494 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918149    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 00:58:04.924494 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918152    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 00:58:04.924494 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918155    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 00:58:04.924494 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918158    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 00:58:04.924494 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918160    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 00:58:04.924494 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918163    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 00:58:04.924494 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918165    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 00:58:04.924494 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918168    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 00:58:04.924494 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918171    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 00:58:04.924494 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918173    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 00:58:04.924960 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918176    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 00:58:04.924960 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918178    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 00:58:04.924960 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918181    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 00:58:04.924960 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918184    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 00:58:04.924960 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918186    2574 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 00:58:04.924960 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918189    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 00:58:04.924960 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:04.918191    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 00:58:04.924960 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.918196    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 00:58:04.924960 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.918293    2574 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 11 00:58:04.924960 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.920113    2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 11 00:58:04.924960 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.921033    2574 server.go:1019] "Starting client certificate rotation"
Sep 11 00:58:04.924960 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.921124    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 00:58:04.924960 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.921161    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 00:58:04.925272 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.923758    2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 00:58:04.928574 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.928554    2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 00:58:04.941158 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.941142    2574 log.go:25] "Validated CRI v1 runtime API"
Sep 11 00:58:04.943947 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.943935    2574 log.go:25] "Validated CRI v1 image API"
Sep 11 00:58:04.945245 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.945225    2574 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 11 00:58:04.949638 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.949610    2574 fs.go:135] Filesystem UUIDs: map[03d3f329-b53e-4590-8d37-24a6d52cb709:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 c63972c9-88e4-495a-872a-5f763bc659cf:/dev/nvme0n1p3]
Sep 11 00:58:04.949704 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.949637    2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 11 00:58:04.951429 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.951406    2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 00:58:04.954511 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.954403    2574 manager.go:216] Machine: {Timestamp:2026-09-11 00:58:04.953200602 +0000 UTC m=+0.350392766 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3105536 MemoryCapacity:32812150784 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29ae00805bd4ead71249fa9c2084eb SystemUUID:ec29ae00-805b-d4ea-d712-49fa9c2084eb BootID:8f5f96cf-f727-48b6-a442-8ce349d0f41d Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406073344 Type:vfs Inodes:4005389 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:70:5f:55:bb:75 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:70:5f:55:bb:75 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1e:49:f2:56:aa:17 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812150784 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 11 00:58:04.954511 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.954503    2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 11 00:58:04.954654 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.954595    2574 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 11 00:58:04.955655 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.955633    2574 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 11 00:58:04.955778 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.955658    2574 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-235.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 11 00:58:04.956295 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.956286    2574 topology_manager.go:138] "Creating topology manager with none policy"
Sep 11 00:58:04.956326 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.956298    2574 container_manager_linux.go:309] "Creating device plugin manager"
Sep 11 00:58:04.956326 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.956306    2574 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 00:58:04.956938 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.956928    2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 00:58:04.957665 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.957656    2574 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 11 00:58:04.961111 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.961100    2574 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 00:58:04.961224 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.961214    2574 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 11 00:58:04.961259 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.961245    2574 kubelet.go:486] "Attempting to sync node with API server"
Sep 11 00:58:04.961259 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.961257    2574 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 11 00:58:04.961310 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.961267    2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 11 00:58:04.961310 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.961276    2574 kubelet.go:398] "Adding apiserver pod source"
Sep 11 00:58:04.961310 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.961284    2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 11 00:58:04.961616 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.961604    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 00:58:04.961651 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.961623    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 00:58:04.963604 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.963585    2574 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 11 00:58:04.965777 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.965763    2574 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 11 00:58:04.965838 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.965793    2574 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 11 00:58:04.967121 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.967110    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 11 00:58:04.967179 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.967126    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 11 00:58:04.967179 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.967133    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 11 00:58:04.967179 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.967138    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 11 00:58:04.967179 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.967146    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 11 00:58:04.967179 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.967151    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 11 00:58:04.967179 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.967165    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 11 00:58:04.967179 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.967171    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 11 00:58:04.967179 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.967177    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 11 00:58:04.967179 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.967182    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 11 00:58:04.967404 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.967190    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 11 00:58:04.967404 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.967197    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 11 00:58:04.967808 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.967798    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 11 00:58:04.967808 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.967807    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 11 00:58:04.969874 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.969852    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lzqvt"
Sep 11 00:58:04.970991 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.970976    2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 11 00:58:04.971070 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:04.971021    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 11 00:58:04.971070 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:04.971053    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 11 00:58:04.971698 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.971686    2574 server.go:1268] "Started kubelet"
Sep 11 00:58:04.971778 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.971756    2574 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 11 00:58:04.971882 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.971831    2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 11 00:58:04.971996 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.971930    2574 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 11 00:58:04.972487 ip-10-0-139-235 systemd[1]: Started Kubernetes Kubelet.
Sep 11 00:58:04.972945 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.972879    2574 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 11 00:58:04.974557 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.974539    2574 server.go:310] "Adding debug handlers to kubelet server"
Sep 11 00:58:04.977268 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:04.977244    2574 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 11 00:58:04.977555 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.977540    2574 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 11 00:58:04.977555 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.977538    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lzqvt"
Sep 11 00:58:04.977555 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.977553    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 11 00:58:04.977706 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:04.977606    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-235.ec2.internal\" not found"
Sep 11 00:58:04.978535 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.978437    2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 11 00:58:04.978645 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.978426    2574 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 11 00:58:04.978645 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.978590    2574 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 11 00:58:04.978757 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.978703    2574 reconstruct.go:97] "Volume reconstruction finished"
Sep 11 00:58:04.979550 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.979533    2574 reconciler.go:29] "Reconciler: start to sync state"
Sep 11 00:58:04.979644 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.979591    2574 factory.go:55] Registering systemd factory
Sep 11 00:58:04.979751 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.979725    2574 factory.go:223] Registration of the systemd container factory successfully
Sep 11 00:58:04.979924 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:04.978803    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-235.ec2.internal.18d41e50f18c7129  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-235.ec2.internal,UID:ip-10-0-139-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-235.ec2.internal,},FirstTimestamp:2026-09-11 00:58:04.971659561 +0000 UTC m=+0.368851725,LastTimestamp:2026-09-11 00:58:04.971659561 +0000 UTC m=+0.368851725,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-235.ec2.internal,}"
Sep 11 00:58:04.980034 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.979975    2574 factory.go:153] Registering CRI-O factory
Sep 11 00:58:04.980034 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.980035    2574 factory.go:223] Registration of the crio container factory successfully
Sep 11 00:58:04.980153 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.980088    2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 11 00:58:04.980153 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.980105    2574 factory.go:103] Registering Raw factory
Sep 11 00:58:04.980153 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.980121    2574 manager.go:1196] Started watching for new ooms in manager
Sep 11 00:58:04.980485 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.980474    2574 manager.go:349] Starting recovery of all containers
Sep 11 00:58:04.986562 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.986538    2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 00:58:04.989250 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:04.989229    2574 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-139-235.ec2.internal\" not found" node="ip-10-0-139-235.ec2.internal"
Sep 11 00:58:04.992505 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.992489    2574 manager.go:354] Recovery completed
Sep 11 00:58:04.993741 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:04.993720    2574 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Sep 11 00:58:04.996447 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.996436    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 00:58:04.998043 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.998027    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-235.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 00:58:04.998119 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.998058    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-235.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 00:58:04.998119 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.998073    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-235.ec2.internal" event="NodeHasSufficientPID"
Sep 11 00:58:04.998621 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.998608    2574 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 11 00:58:04.998699 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.998620    2574 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 11 00:58:04.998699 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.998626    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 00:58:04.998699 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.998642    2574 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 00:58:04.999410 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.999388    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-235.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 00:58:04.999464 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.999420    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-235.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 00:58:04.999464 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:04.999431    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-235.ec2.internal" event="NodeHasSufficientPID"
Sep 11 00:58:05.000670 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.000657    2574 policy_none.go:49] "None policy: Start"
Sep 11 00:58:05.000738 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.000677    2574 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 11 00:58:05.000738 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.000690    2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 11 00:58:05.002559 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.002546    2574 policy_none.go:47] "Start"
Sep 11 00:58:05.041257 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.040562    2574 manager.go:337] "Starting Device Plugin manager"
Sep 11 00:58:05.041257 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:05.040629    2574 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 11 00:58:05.041257 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.040643    2574 server.go:87] "Starting device plugin registration server"
Sep 11 00:58:05.041257 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.040873    2574 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 11 00:58:05.041257 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.040885    2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 11 00:58:05.041257 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.041123    2574 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 11 00:58:05.041257 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.041531    2574 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 11 00:58:05.041257 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.041541    2574 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 11 00:58:05.065063 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:05.044986    2574 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 11 00:58:05.065063 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:05.045033    2574 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-235.ec2.internal\" not found"
Sep 11 00:58:05.092388 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.092355    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 11 00:58:05.093679 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.093663    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 11 00:58:05.093766 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.093707    2574 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 11 00:58:05.093766 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.093728    2574 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 11 00:58:05.093842 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:05.093765    2574 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 11 00:58:05.096093 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.096076    2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 00:58:05.141984 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.141965    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 00:58:05.142746 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.142731    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-235.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 00:58:05.142802 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.142758    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-235.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 00:58:05.142802 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.142768    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-235.ec2.internal" event="NodeHasSufficientPID"
Sep 11 00:58:05.142802 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.142787    2574 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-235.ec2.internal"
Sep 11 00:58:05.148739 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.148725    2574 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-235.ec2.internal"
Sep 11 00:58:05.148780 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:05.148743    2574 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-235.ec2.internal\": node \"ip-10-0-139-235.ec2.internal\" not found"
Sep 11 00:58:05.166051 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:05.166035    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-235.ec2.internal\" not found"
Sep 11 00:58:05.194363 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.194344    2574 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-235.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-235.ec2.internal"]
Sep 11 00:58:05.194415 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.194401    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 00:58:05.195636 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.195621    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-235.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 00:58:05.195705 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.195650    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-235.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 00:58:05.195705 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.195659    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-235.ec2.internal" event="NodeHasSufficientPID"
Sep 11 00:58:05.197714 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.197702    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 00:58:05.198240 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.198220    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-235.ec2.internal"
Sep 11 00:58:05.198356 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.198258    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 00:58:05.198356 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.198330    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-235.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 00:58:05.198356 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.198353    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-235.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 00:58:05.198478 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.198365    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-235.ec2.internal" event="NodeHasSufficientPID"
Sep 11 00:58:05.198986 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.198971    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-235.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 00:58:05.199069 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.198997    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-235.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 00:58:05.199069 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.199006    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-235.ec2.internal" event="NodeHasSufficientPID"
Sep 11 00:58:05.200414 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.200402    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-235.ec2.internal"
Sep 11 00:58:05.200464 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.200425    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 00:58:05.201223 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.201207    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-235.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 00:58:05.201311 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.201231    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-235.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 00:58:05.201311 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.201244    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-235.ec2.internal" event="NodeHasSufficientPID"
Sep 11 00:58:05.229490 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:05.229465    2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-235.ec2.internal\" not found" node="ip-10-0-139-235.ec2.internal"
Sep 11 00:58:05.233840 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:05.233825    2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-235.ec2.internal\" not found" node="ip-10-0-139-235.ec2.internal"
Sep 11 00:58:05.267106 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:05.267085    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-235.ec2.internal\" not found"
Sep 11 00:58:05.281000 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.280974    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d958aa85aad010b15ac136386e20db34-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-235.ec2.internal\" (UID: \"d958aa85aad010b15ac136386e20db34\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-235.ec2.internal"
Sep 11 00:58:05.281091 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.281008    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9f1f5adc45e045b22e89459332dfb6a0-config\") pod \"kube-apiserver-proxy-ip-10-0-139-235.ec2.internal\" (UID: \"9f1f5adc45e045b22e89459332dfb6a0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-235.ec2.internal"
Sep 11 00:58:05.281091 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.281033    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d958aa85aad010b15ac136386e20db34-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-235.ec2.internal\" (UID: \"d958aa85aad010b15ac136386e20db34\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-235.ec2.internal"
Sep 11 00:58:05.367433 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:05.367401    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-235.ec2.internal\" not found"
Sep 11 00:58:05.381750 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.381730    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d958aa85aad010b15ac136386e20db34-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-235.ec2.internal\" (UID: \"d958aa85aad010b15ac136386e20db34\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-235.ec2.internal"
Sep 11 00:58:05.381813 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.381755    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d958aa85aad010b15ac136386e20db34-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-235.ec2.internal\" (UID: \"d958aa85aad010b15ac136386e20db34\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-235.ec2.internal"
Sep 11 00:58:05.381813 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.381771    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9f1f5adc45e045b22e89459332dfb6a0-config\") pod \"kube-apiserver-proxy-ip-10-0-139-235.ec2.internal\" (UID: \"9f1f5adc45e045b22e89459332dfb6a0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-235.ec2.internal"
Sep 11 00:58:05.381813 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.381806    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9f1f5adc45e045b22e89459332dfb6a0-config\") pod \"kube-apiserver-proxy-ip-10-0-139-235.ec2.internal\" (UID: \"9f1f5adc45e045b22e89459332dfb6a0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-235.ec2.internal"
Sep 11 00:58:05.381957 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.381824    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d958aa85aad010b15ac136386e20db34-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-235.ec2.internal\" (UID: \"d958aa85aad010b15ac136386e20db34\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-235.ec2.internal"
Sep 11 00:58:05.381957 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.381848    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d958aa85aad010b15ac136386e20db34-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-235.ec2.internal\" (UID: \"d958aa85aad010b15ac136386e20db34\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-235.ec2.internal"
Sep 11 00:58:05.468147 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:05.468075    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-235.ec2.internal\" not found"
Sep 11 00:58:05.531594 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.531571    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-235.ec2.internal"
Sep 11 00:58:05.536528 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.536263    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-235.ec2.internal"
Sep 11 00:58:05.568590 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:05.568458    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-235.ec2.internal\" not found"
Sep 11 00:58:05.668950 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:05.668928    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-235.ec2.internal\" not found"
Sep 11 00:58:05.769520 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:05.769471    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-235.ec2.internal\" not found"
Sep 11 00:58:05.803552 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.803527    2574 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 00:58:05.878391 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.878371    2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-235.ec2.internal"
Sep 11 00:58:05.889550 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.889533    2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 00:58:05.891425 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.891413    2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-235.ec2.internal"
Sep 11 00:58:05.899218 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.899199    2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 00:58:05.921333 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.921311    2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 11 00:58:05.921721 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.921429    2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 00:58:05.921721 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.921456    2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 00:58:05.921721 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.921504    2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 00:58:05.961630 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.961613    2574 apiserver.go:52] "Watching apiserver"
Sep 11 00:58:05.978295 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.978272    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 11 00:58:05.979217 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.979201    2574 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 11 00:58:05.980109 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.980091    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-jbpsj","kube-system/konnectivity-agent-l5nhz","kube-system/kube-apiserver-proxy-ip-10-0-139-235.ec2.internal","openshift-cluster-node-tuning-operator/tuned-8q8bw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-235.ec2.internal","openshift-network-operator/iptables-alerter-qx27q","openshift-ovn-kubernetes/ovnkube-node-d7crj","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhmsk","openshift-dns/node-resolver-zw22q","openshift-image-registry/node-ca-hh5wm","openshift-multus/multus-additional-cni-plugins-7j9mn","openshift-multus/multus-w9q8q","openshift-multus/network-metrics-daemon-9s5p9","openshift-network-diagnostics/network-check-target-9hl4x"]
Sep 11 00:58:05.981714 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.981693    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 00:53:05 +0000 UTC" deadline="2028-06-02 08:48:04.096798964 +0000 UTC"
Sep 11 00:58:05.981760 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.981713    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15127h49m58.115088276s"
Sep 11 00:58:05.983170 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.983156    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:05.985189 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.985169    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 11 00:58:05.985284 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.985235    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 11 00:58:05.985344 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.985298    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 11 00:58:05.985344 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.985339    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 11 00:58:05.985455 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.985425    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8fv6z\""
Sep 11 00:58:05.985511 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.985492    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 11 00:58:05.985560 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.985549    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 11 00:58:05.985804 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.985787    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7j9mn"
Sep 11 00:58:05.987382 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.987362    2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 00:58:05.987513 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.987496    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 11 00:58:05.987572 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.987555    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-l2lq6\""
Sep 11 00:58:05.987702 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.987687    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 11 00:58:05.987761 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.987741    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 11 00:58:05.987834 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.987817    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 11 00:58:05.987884 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.987854    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 11 00:58:05.988411 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.988394    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hl4x"
Sep 11 00:58:05.988713 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:05.988650    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hl4x" podUID="526304d7-8bd3-4196-a695-f5845e8846a1"
Sep 11 00:58:05.991345 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.991326    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:05.993164 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.993139    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 11 00:58:05.993274 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.993256    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ch2rg\""
Sep 11 00:58:05.993364 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.993348    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 11 00:58:05.993827 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.993814    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhmsk"
Sep 11 00:58:05.995745 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.995724    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 11 00:58:05.995745 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.995739    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vgk7n\""
Sep 11 00:58:05.995883 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.995832    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 11 00:58:05.996080 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.996064    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 11 00:58:05.996575 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.996563    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qx27q"
Sep 11 00:58:05.998327 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.998310    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 11 00:58:05.998678 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.998666    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 11 00:58:05.998738 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.998724    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qjflx\""
Sep 11 00:58:05.998789 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.998776    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 11 00:58:05.999124 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:05.999110    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zw22q"
Sep 11 00:58:06.000698 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.000684    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 11 00:58:06.000951 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.000933    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 11 00:58:06.001054 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.001018    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-td8k7\""
Sep 11 00:58:06.002133 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.002118    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5p9"
Sep 11 00:58:06.002198 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:06.002183    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9s5p9" podUID="77093864-53b6-4044-952a-d8f1a965dd1d"
Sep 11 00:58:06.004661 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.004646    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l5nhz"
Sep 11 00:58:06.006898 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.006881    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 11 00:58:06.006898 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.006896    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-ztsqp\""
Sep 11 00:58:06.007095 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.007078    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 11 00:58:06.007612 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.007597    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hh5wm"
Sep 11 00:58:06.009359 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.009342    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 11 00:58:06.009606 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.009588    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 11 00:58:06.009606 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.009590    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 11 00:58:06.009731 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.009615    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-wwmd8\""
Sep 11 00:58:06.010465 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.010449    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:58:06.010550 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:06.010530    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbpsj" podUID="ff3ee2a6-e101-4cde-8d89-489f1db81ab8"
Sep 11 00:58:06.011782 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.011770    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pd6n4"
Sep 11 00:58:06.012792 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.012777    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.014977 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.014854    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 11 00:58:06.014977 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.014938    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-65s48\""
Sep 11 00:58:06.020693 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.020652    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pd6n4"
Sep 11 00:58:06.079221 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.079203    2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 11 00:58:06.084347 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.084328    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glssd\" (UniqueName: \"kubernetes.io/projected/8ad98990-55b3-4499-9b32-a8b394175160-kube-api-access-glssd\") pod \"aws-ebs-csi-driver-node-zhmsk\" (UID: \"8ad98990-55b3-4499-9b32-a8b394175160\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhmsk"
Sep 11 00:58:06.084441 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.084356    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-hostroot\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.084441 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.084383    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-run-systemd\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.084441 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.084396    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/390448c5-9242-4562-84e5-0307125ba1ad-etc-systemd\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.084441 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.084410    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8ad98990-55b3-4499-9b32-a8b394175160-sys-fs\") pod \"aws-ebs-csi-driver-node-zhmsk\" (UID: \"8ad98990-55b3-4499-9b32-a8b394175160\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhmsk"
Sep 11 00:58:06.084441 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.084430    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-etc-kubernetes\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.084602 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.084450    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ff3ee2a6-e101-4cde-8d89-489f1db81ab8-dbus\") pod \"global-pull-secret-syncer-jbpsj\" (UID: \"ff3ee2a6-e101-4cde-8d89-489f1db81ab8\") " pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:58:06.084602 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.084493    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8ad98990-55b3-4499-9b32-a8b394175160-registration-dir\") pod \"aws-ebs-csi-driver-node-zhmsk\" (UID: \"8ad98990-55b3-4499-9b32-a8b394175160\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhmsk"
Sep 11 00:58:06.084602 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.084518    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-host-var-lib-kubelet\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.084602 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.084540    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-multus-conf-dir\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.084602 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.084563    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-multus-daemon-config\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.084602 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.084589    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-host-cni-bin\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.084814 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.084615    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5npk\" (UniqueName: \"kubernetes.io/projected/5176ce4f-e447-4f42-a338-2f1ee4761210-kube-api-access-f5npk\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.084814 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.084637    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d097ee33-8900-43b7-9eae-35ac88373d1d-hosts-file\") pod \"node-resolver-zw22q\" (UID: \"d097ee33-8900-43b7-9eae-35ac88373d1d\") " pod="openshift-dns/node-resolver-zw22q"
Sep 11 00:58:06.084814 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.084669    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1fe146d4-3a94-48d2-b383-ea0a4c301b54-cnibin\") pod \"multus-additional-cni-plugins-7j9mn\" (UID: \"1fe146d4-3a94-48d2-b383-ea0a4c301b54\") " pod="openshift-multus/multus-additional-cni-plugins-7j9mn"
Sep 11 00:58:06.084814 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.084694    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/390448c5-9242-4562-84e5-0307125ba1ad-etc-tuned\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.084814 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.084718    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-host-run-k8s-cni-cncf-io\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.084814 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.084742    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-host-var-lib-cni-bin\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.084814 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.084766    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-etc-openvswitch\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.084814 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.084791    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66bcw\" (UniqueName: \"kubernetes.io/projected/1625cd1f-5711-4d05-9cfc-74a2a2c1c637-kube-api-access-66bcw\") pod \"node-ca-hh5wm\" (UID: \"1625cd1f-5711-4d05-9cfc-74a2a2c1c637\") " pod="openshift-image-registry/node-ca-hh5wm"
Sep 11 00:58:06.084814 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.084814    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/390448c5-9242-4562-84e5-0307125ba1ad-sys\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.085345 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.084839    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8ad98990-55b3-4499-9b32-a8b394175160-etc-selinux\") pod \"aws-ebs-csi-driver-node-zhmsk\" (UID: \"8ad98990-55b3-4499-9b32-a8b394175160\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhmsk"
Sep 11 00:58:06.085345 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.084878    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-system-cni-dir\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.085345 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.084957    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-cni-binary-copy\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.085345 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.084995    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-node-log\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.085345 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085021    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5176ce4f-e447-4f42-a338-2f1ee4761210-ovnkube-config\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.085345 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085044    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5176ce4f-e447-4f42-a338-2f1ee4761210-ovnkube-script-lib\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.085345 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085066    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1625cd1f-5711-4d05-9cfc-74a2a2c1c637-serviceca\") pod \"node-ca-hh5wm\" (UID: \"1625cd1f-5711-4d05-9cfc-74a2a2c1c637\") " pod="openshift-image-registry/node-ca-hh5wm"
Sep 11 00:58:06.085345 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085090    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff3ee2a6-e101-4cde-8d89-489f1db81ab8-original-pull-secret\") pod \"global-pull-secret-syncer-jbpsj\" (UID: \"ff3ee2a6-e101-4cde-8d89-489f1db81ab8\") " pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:58:06.085345 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085135    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8ad98990-55b3-4499-9b32-a8b394175160-socket-dir\") pod \"aws-ebs-csi-driver-node-zhmsk\" (UID: \"8ad98990-55b3-4499-9b32-a8b394175160\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhmsk"
Sep 11 00:58:06.085345 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085182    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/390448c5-9242-4562-84e5-0307125ba1ad-etc-sysctl-d\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.085345 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085205    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-host-cni-netd\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.085345 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085229    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5176ce4f-e447-4f42-a338-2f1ee4761210-ovn-node-metrics-cert\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.085345 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085283    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/390448c5-9242-4562-84e5-0307125ba1ad-etc-modprobe-d\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.085345 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085320    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/396a7cf9-e334-407d-ad45-789fb8dfba05-konnectivity-ca\") pod \"konnectivity-agent-l5nhz\" (UID: \"396a7cf9-e334-407d-ad45-789fb8dfba05\") " pod="kube-system/konnectivity-agent-l5nhz"
Sep 11 00:58:06.085870 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085357    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ff3ee2a6-e101-4cde-8d89-489f1db81ab8-kubelet-config\") pod \"global-pull-secret-syncer-jbpsj\" (UID: \"ff3ee2a6-e101-4cde-8d89-489f1db81ab8\") " pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:58:06.085870 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085380    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-multus-cni-dir\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.085870 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085412    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-host-run-multus-certs\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.085870 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085443    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-var-lib-openvswitch\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.085870 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085466    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/396a7cf9-e334-407d-ad45-789fb8dfba05-agent-certs\") pod \"konnectivity-agent-l5nhz\" (UID: \"396a7cf9-e334-407d-ad45-789fb8dfba05\") " pod="kube-system/konnectivity-agent-l5nhz"
Sep 11 00:58:06.085870 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085488    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77093864-53b6-4044-952a-d8f1a965dd1d-metrics-certs\") pod \"network-metrics-daemon-9s5p9\" (UID: \"77093864-53b6-4044-952a-d8f1a965dd1d\") " pod="openshift-multus/network-metrics-daemon-9s5p9"
Sep 11 00:58:06.085870 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085521    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-host-slash\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.085870 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085543    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-run-openvswitch\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.085870 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085565    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tdws\" (UniqueName: \"kubernetes.io/projected/526304d7-8bd3-4196-a695-f5845e8846a1-kube-api-access-2tdws\") pod \"network-check-target-9hl4x\" (UID: \"526304d7-8bd3-4196-a695-f5845e8846a1\") " pod="openshift-network-diagnostics/network-check-target-9hl4x"
Sep 11 00:58:06.085870 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085585    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1fe146d4-3a94-48d2-b383-ea0a4c301b54-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7j9mn\" (UID: \"1fe146d4-3a94-48d2-b383-ea0a4c301b54\") " pod="openshift-multus/multus-additional-cni-plugins-7j9mn"
Sep 11 00:58:06.085870 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085599    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/390448c5-9242-4562-84e5-0307125ba1ad-etc-kubernetes\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.085870 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085615    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/390448c5-9242-4562-84e5-0307125ba1ad-etc-sysctl-conf\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.085870 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085632    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/390448c5-9242-4562-84e5-0307125ba1ad-run\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.085870 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085647    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5176ce4f-e447-4f42-a338-2f1ee4761210-env-overrides\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.085870 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085661    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wh9kz\" (UniqueName: \"kubernetes.io/projected/c29dcdd7-781d-411d-b6e1-4cb59de9be56-kube-api-access-wh9kz\") pod \"iptables-alerter-qx27q\" (UID: \"c29dcdd7-781d-411d-b6e1-4cb59de9be56\") " pod="openshift-network-operator/iptables-alerter-qx27q"
Sep 11 00:58:06.085870 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085680    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1fe146d4-3a94-48d2-b383-ea0a4c301b54-system-cni-dir\") pod \"multus-additional-cni-plugins-7j9mn\" (UID: \"1fe146d4-3a94-48d2-b383-ea0a4c301b54\") " pod="openshift-multus/multus-additional-cni-plugins-7j9mn"
Sep 11 00:58:06.086499 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085700    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trqzg\" (UniqueName: \"kubernetes.io/projected/77093864-53b6-4044-952a-d8f1a965dd1d-kube-api-access-trqzg\") pod \"network-metrics-daemon-9s5p9\" (UID: \"77093864-53b6-4044-952a-d8f1a965dd1d\") " pod="openshift-multus/network-metrics-daemon-9s5p9"
Sep 11 00:58:06.086499 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085715    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8lzw\" (UniqueName: \"kubernetes.io/projected/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-kube-api-access-h8lzw\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.086499 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085729    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-log-socket\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.086499 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085745    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.086499 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085768    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c29dcdd7-781d-411d-b6e1-4cb59de9be56-iptables-alerter-script\") pod \"iptables-alerter-qx27q\" (UID: \"c29dcdd7-781d-411d-b6e1-4cb59de9be56\") " pod="openshift-network-operator/iptables-alerter-qx27q"
Sep 11 00:58:06.086499 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085795    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/390448c5-9242-4562-84e5-0307125ba1ad-tmp\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.086499 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085818    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-cnibin\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.086499 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085838    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-host-run-netns\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.086499 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085867    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-host-run-netns\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.086499 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085892    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-run-ovn\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.086499 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085928    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d097ee33-8900-43b7-9eae-35ac88373d1d-tmp-dir\") pod \"node-resolver-zw22q\" (UID: \"d097ee33-8900-43b7-9eae-35ac88373d1d\") " pod="openshift-dns/node-resolver-zw22q"
Sep 11 00:58:06.086499 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085945    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1625cd1f-5711-4d05-9cfc-74a2a2c1c637-host\") pod \"node-ca-hh5wm\" (UID: \"1625cd1f-5711-4d05-9cfc-74a2a2c1c637\") " pod="openshift-image-registry/node-ca-hh5wm"
Sep 11 00:58:06.086499 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085959    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mp5x4\" (UniqueName: \"kubernetes.io/projected/1fe146d4-3a94-48d2-b383-ea0a4c301b54-kube-api-access-mp5x4\") pod \"multus-additional-cni-plugins-7j9mn\" (UID: \"1fe146d4-3a94-48d2-b383-ea0a4c301b54\") " pod="openshift-multus/multus-additional-cni-plugins-7j9mn"
Sep 11 00:58:06.086499 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085973    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8ad98990-55b3-4499-9b32-a8b394175160-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zhmsk\" (UID: \"8ad98990-55b3-4499-9b32-a8b394175160\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhmsk"
Sep 11 00:58:06.086499 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.085986    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-host-var-lib-cni-multus\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.086499 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.086001    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/390448c5-9242-4562-84e5-0307125ba1ad-lib-modules\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.087000 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.086036    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/390448c5-9242-4562-84e5-0307125ba1ad-host\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.087000 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.086055    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c29dcdd7-781d-411d-b6e1-4cb59de9be56-host-slash\") pod \"iptables-alerter-qx27q\" (UID: \"c29dcdd7-781d-411d-b6e1-4cb59de9be56\") " pod="openshift-network-operator/iptables-alerter-qx27q"
Sep 11 00:58:06.087000 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.086070    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xkwk\" (UniqueName: \"kubernetes.io/projected/d097ee33-8900-43b7-9eae-35ac88373d1d-kube-api-access-4xkwk\") pod \"node-resolver-zw22q\" (UID: \"d097ee33-8900-43b7-9eae-35ac88373d1d\") " pod="openshift-dns/node-resolver-zw22q"
Sep 11 00:58:06.087000 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.086092    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1fe146d4-3a94-48d2-b383-ea0a4c301b54-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7j9mn\" (UID: \"1fe146d4-3a94-48d2-b383-ea0a4c301b54\") " pod="openshift-multus/multus-additional-cni-plugins-7j9mn"
Sep 11 00:58:06.087000 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.086107    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-multus-socket-dir-parent\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.087000 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.086121    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-host-kubelet\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.087000 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.086135    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1fe146d4-3a94-48d2-b383-ea0a4c301b54-cni-binary-copy\") pod \"multus-additional-cni-plugins-7j9mn\" (UID: \"1fe146d4-3a94-48d2-b383-ea0a4c301b54\") " pod="openshift-multus/multus-additional-cni-plugins-7j9mn"
Sep 11 00:58:06.087000 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.086154    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1fe146d4-3a94-48d2-b383-ea0a4c301b54-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7j9mn\" (UID: \"1fe146d4-3a94-48d2-b383-ea0a4c301b54\") " pod="openshift-multus/multus-additional-cni-plugins-7j9mn"
Sep 11 00:58:06.087000 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.086169    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/390448c5-9242-4562-84e5-0307125ba1ad-var-lib-kubelet\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.087000 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.086181    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdndj\" (UniqueName: \"kubernetes.io/projected/390448c5-9242-4562-84e5-0307125ba1ad-kube-api-access-gdndj\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.087000 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.086217    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-systemd-units\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.087000 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.086252    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-host-run-ovn-kubernetes\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.087000 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.086292    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/390448c5-9242-4562-84e5-0307125ba1ad-etc-sysconfig\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.087000 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.086325    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8ad98990-55b3-4499-9b32-a8b394175160-device-dir\") pod \"aws-ebs-csi-driver-node-zhmsk\" (UID: \"8ad98990-55b3-4499-9b32-a8b394175160\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhmsk"
Sep 11 00:58:06.148072 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:06.148044    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd958aa85aad010b15ac136386e20db34.slice/crio-ab9b61f53be03f21cc566b10b78f85b15e04946c32a067b34f36b2ffff035448 WatchSource:0}: Error finding container ab9b61f53be03f21cc566b10b78f85b15e04946c32a067b34f36b2ffff035448: Status 404 returned error can't find the container with id ab9b61f53be03f21cc566b10b78f85b15e04946c32a067b34f36b2ffff035448
Sep 11 00:58:06.148257 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:06.148246    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f1f5adc45e045b22e89459332dfb6a0.slice/crio-fe75795b91ed4bc4e6a78de66736b62e4dbec5e3227d3f8bdcbbc7fd50eaecf1 WatchSource:0}: Error finding container fe75795b91ed4bc4e6a78de66736b62e4dbec5e3227d3f8bdcbbc7fd50eaecf1: Status 404 returned error can't find the container with id fe75795b91ed4bc4e6a78de66736b62e4dbec5e3227d3f8bdcbbc7fd50eaecf1
Sep 11 00:58:06.154123 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.154108    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 00:58:06.187090 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187070    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trqzg\" (UniqueName: \"kubernetes.io/projected/77093864-53b6-4044-952a-d8f1a965dd1d-kube-api-access-trqzg\") pod \"network-metrics-daemon-9s5p9\" (UID: \"77093864-53b6-4044-952a-d8f1a965dd1d\") " pod="openshift-multus/network-metrics-daemon-9s5p9"
Sep 11 00:58:06.187179 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187096    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8lzw\" (UniqueName: \"kubernetes.io/projected/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-kube-api-access-h8lzw\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.187179 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187112    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-log-socket\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.187179 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187128    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.187179 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187142    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c29dcdd7-781d-411d-b6e1-4cb59de9be56-iptables-alerter-script\") pod \"iptables-alerter-qx27q\" (UID: \"c29dcdd7-781d-411d-b6e1-4cb59de9be56\") " pod="openshift-network-operator/iptables-alerter-qx27q"
Sep 11 00:58:06.187179 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187156    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/390448c5-9242-4562-84e5-0307125ba1ad-tmp\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.187179 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187172    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-cnibin\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.187434 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187194    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-host-run-netns\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.187434 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187212    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.187434 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187218    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-host-run-netns\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.187434 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187251    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-log-socket\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.187434 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187272    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-host-run-netns\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.187434 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187276    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-run-ovn\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.187434 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187302    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d097ee33-8900-43b7-9eae-35ac88373d1d-tmp-dir\") pod \"node-resolver-zw22q\" (UID: \"d097ee33-8900-43b7-9eae-35ac88373d1d\") " pod="openshift-dns/node-resolver-zw22q"
Sep 11 00:58:06.187434 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187310    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-host-run-netns\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.187434 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187321    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-cnibin\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.187434 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187359    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-run-ovn\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.187434 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187363    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1625cd1f-5711-4d05-9cfc-74a2a2c1c637-host\") pod \"node-ca-hh5wm\" (UID: \"1625cd1f-5711-4d05-9cfc-74a2a2c1c637\") " pod="openshift-image-registry/node-ca-hh5wm"
Sep 11 00:58:06.187434 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187403    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mp5x4\" (UniqueName: \"kubernetes.io/projected/1fe146d4-3a94-48d2-b383-ea0a4c301b54-kube-api-access-mp5x4\") pod \"multus-additional-cni-plugins-7j9mn\" (UID: \"1fe146d4-3a94-48d2-b383-ea0a4c301b54\") " pod="openshift-multus/multus-additional-cni-plugins-7j9mn"
Sep 11 00:58:06.187434 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187421    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1625cd1f-5711-4d05-9cfc-74a2a2c1c637-host\") pod \"node-ca-hh5wm\" (UID: \"1625cd1f-5711-4d05-9cfc-74a2a2c1c637\") " pod="openshift-image-registry/node-ca-hh5wm"
Sep 11 00:58:06.187876 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187440    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8ad98990-55b3-4499-9b32-a8b394175160-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zhmsk\" (UID: \"8ad98990-55b3-4499-9b32-a8b394175160\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhmsk"
Sep 11 00:58:06.187876 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187463    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-host-var-lib-cni-multus\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.187876 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187485    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/390448c5-9242-4562-84e5-0307125ba1ad-lib-modules\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.187876 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187501    2574 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 11 00:58:06.187876 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187522    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8ad98990-55b3-4499-9b32-a8b394175160-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zhmsk\" (UID: \"8ad98990-55b3-4499-9b32-a8b394175160\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhmsk"
Sep 11 00:58:06.187876 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187546    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/390448c5-9242-4562-84e5-0307125ba1ad-host\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.187876 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187567    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-host-var-lib-cni-multus\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.187876 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187506    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/390448c5-9242-4562-84e5-0307125ba1ad-host\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.187876 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187597    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d097ee33-8900-43b7-9eae-35ac88373d1d-tmp-dir\") pod \"node-resolver-zw22q\" (UID: \"d097ee33-8900-43b7-9eae-35ac88373d1d\") " pod="openshift-dns/node-resolver-zw22q"
Sep 11 00:58:06.187876 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187623    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c29dcdd7-781d-411d-b6e1-4cb59de9be56-host-slash\") pod \"iptables-alerter-qx27q\" (UID: \"c29dcdd7-781d-411d-b6e1-4cb59de9be56\") " pod="openshift-network-operator/iptables-alerter-qx27q"
Sep 11 00:58:06.187876 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187600    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c29dcdd7-781d-411d-b6e1-4cb59de9be56-host-slash\") pod \"iptables-alerter-qx27q\" (UID: \"c29dcdd7-781d-411d-b6e1-4cb59de9be56\") " pod="openshift-network-operator/iptables-alerter-qx27q"
Sep 11 00:58:06.187876 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187666    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xkwk\" (UniqueName: \"kubernetes.io/projected/d097ee33-8900-43b7-9eae-35ac88373d1d-kube-api-access-4xkwk\") pod \"node-resolver-zw22q\" (UID: \"d097ee33-8900-43b7-9eae-35ac88373d1d\") " pod="openshift-dns/node-resolver-zw22q"
Sep 11 00:58:06.187876 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187684    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/390448c5-9242-4562-84e5-0307125ba1ad-lib-modules\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.187876 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187703    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1fe146d4-3a94-48d2-b383-ea0a4c301b54-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7j9mn\" (UID: \"1fe146d4-3a94-48d2-b383-ea0a4c301b54\") " pod="openshift-multus/multus-additional-cni-plugins-7j9mn"
Sep 11 00:58:06.187876 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187728    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-multus-socket-dir-parent\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.187876 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187753    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-host-kubelet\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.187876 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187759    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c29dcdd7-781d-411d-b6e1-4cb59de9be56-iptables-alerter-script\") pod \"iptables-alerter-qx27q\" (UID: \"c29dcdd7-781d-411d-b6e1-4cb59de9be56\") " pod="openshift-network-operator/iptables-alerter-qx27q"
Sep 11 00:58:06.187876 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187775    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1fe146d4-3a94-48d2-b383-ea0a4c301b54-cni-binary-copy\") pod \"multus-additional-cni-plugins-7j9mn\" (UID: \"1fe146d4-3a94-48d2-b383-ea0a4c301b54\") " pod="openshift-multus/multus-additional-cni-plugins-7j9mn"
Sep 11 00:58:06.188621 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187795    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1fe146d4-3a94-48d2-b383-ea0a4c301b54-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7j9mn\" (UID: \"1fe146d4-3a94-48d2-b383-ea0a4c301b54\") " pod="openshift-multus/multus-additional-cni-plugins-7j9mn"
Sep 11 00:58:06.188621 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187817    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/390448c5-9242-4562-84e5-0307125ba1ad-var-lib-kubelet\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.188621 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187828    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-host-kubelet\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.188621 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187869    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1fe146d4-3a94-48d2-b383-ea0a4c301b54-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7j9mn\" (UID: \"1fe146d4-3a94-48d2-b383-ea0a4c301b54\") " pod="openshift-multus/multus-additional-cni-plugins-7j9mn"
Sep 11 00:58:06.188621 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187873    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-multus-socket-dir-parent\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.188621 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187887    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/390448c5-9242-4562-84e5-0307125ba1ad-var-lib-kubelet\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.188621 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187932    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdndj\" (UniqueName: \"kubernetes.io/projected/390448c5-9242-4562-84e5-0307125ba1ad-kube-api-access-gdndj\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.188621 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187952    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-systemd-units\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.188621 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187966    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-host-run-ovn-kubernetes\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.188621 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.187982    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/390448c5-9242-4562-84e5-0307125ba1ad-etc-sysconfig\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.188621 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188014    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-systemd-units\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.188621 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188021    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8ad98990-55b3-4499-9b32-a8b394175160-device-dir\") pod \"aws-ebs-csi-driver-node-zhmsk\" (UID: \"8ad98990-55b3-4499-9b32-a8b394175160\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhmsk"
Sep 11 00:58:06.188621 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188030    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/390448c5-9242-4562-84e5-0307125ba1ad-etc-sysconfig\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.188621 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188045    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glssd\" (UniqueName: \"kubernetes.io/projected/8ad98990-55b3-4499-9b32-a8b394175160-kube-api-access-glssd\") pod \"aws-ebs-csi-driver-node-zhmsk\" (UID: \"8ad98990-55b3-4499-9b32-a8b394175160\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhmsk"
Sep 11 00:58:06.188621 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188061    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-host-run-ovn-kubernetes\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.188621 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188066    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-hostroot\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.188621 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188093    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-run-systemd\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.189600 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188116    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/390448c5-9242-4562-84e5-0307125ba1ad-etc-systemd\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.189600 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188140    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8ad98990-55b3-4499-9b32-a8b394175160-sys-fs\") pod \"aws-ebs-csi-driver-node-zhmsk\" (UID: \"8ad98990-55b3-4499-9b32-a8b394175160\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhmsk"
Sep 11 00:58:06.189600 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188169    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-hostroot\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.189600 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188117    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8ad98990-55b3-4499-9b32-a8b394175160-device-dir\") pod \"aws-ebs-csi-driver-node-zhmsk\" (UID: \"8ad98990-55b3-4499-9b32-a8b394175160\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhmsk"
Sep 11 00:58:06.189600 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188186    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-etc-kubernetes\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.189600 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188212    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-run-systemd\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.189600 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188233    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-etc-kubernetes\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.189600 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188237    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/390448c5-9242-4562-84e5-0307125ba1ad-etc-systemd\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.189600 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188264    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8ad98990-55b3-4499-9b32-a8b394175160-sys-fs\") pod \"aws-ebs-csi-driver-node-zhmsk\" (UID: \"8ad98990-55b3-4499-9b32-a8b394175160\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhmsk"
Sep 11 00:58:06.189600 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188268    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ff3ee2a6-e101-4cde-8d89-489f1db81ab8-dbus\") pod \"global-pull-secret-syncer-jbpsj\" (UID: \"ff3ee2a6-e101-4cde-8d89-489f1db81ab8\") " pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:58:06.189600 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188289    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8ad98990-55b3-4499-9b32-a8b394175160-registration-dir\") pod \"aws-ebs-csi-driver-node-zhmsk\" (UID: \"8ad98990-55b3-4499-9b32-a8b394175160\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhmsk"
Sep 11 00:58:06.189600 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188302    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-host-var-lib-kubelet\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.189600 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188320    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-multus-conf-dir\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.189600 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188336    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-multus-daemon-config\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.189600 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188341    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1fe146d4-3a94-48d2-b383-ea0a4c301b54-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7j9mn\" (UID: \"1fe146d4-3a94-48d2-b383-ea0a4c301b54\") " pod="openshift-multus/multus-additional-cni-plugins-7j9mn"
Sep 11 00:58:06.189600 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188341    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1fe146d4-3a94-48d2-b383-ea0a4c301b54-cni-binary-copy\") pod \"multus-additional-cni-plugins-7j9mn\" (UID: \"1fe146d4-3a94-48d2-b383-ea0a4c301b54\") " pod="openshift-multus/multus-additional-cni-plugins-7j9mn"
Sep 11 00:58:06.189600 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188358    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-host-cni-bin\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.190481 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188383    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5npk\" (UniqueName: \"kubernetes.io/projected/5176ce4f-e447-4f42-a338-2f1ee4761210-kube-api-access-f5npk\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.190481 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188384    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-host-var-lib-kubelet\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.190481 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188387    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-multus-conf-dir\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.190481 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188378    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8ad98990-55b3-4499-9b32-a8b394175160-registration-dir\") pod \"aws-ebs-csi-driver-node-zhmsk\" (UID: \"8ad98990-55b3-4499-9b32-a8b394175160\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhmsk"
Sep 11 00:58:06.190481 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188404    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d097ee33-8900-43b7-9eae-35ac88373d1d-hosts-file\") pod \"node-resolver-zw22q\" (UID: \"d097ee33-8900-43b7-9eae-35ac88373d1d\") " pod="openshift-dns/node-resolver-zw22q"
Sep 11 00:58:06.190481 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188409    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-host-cni-bin\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.190481 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188442    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1fe146d4-3a94-48d2-b383-ea0a4c301b54-cnibin\") pod \"multus-additional-cni-plugins-7j9mn\" (UID: \"1fe146d4-3a94-48d2-b383-ea0a4c301b54\") " pod="openshift-multus/multus-additional-cni-plugins-7j9mn"
Sep 11 00:58:06.190481 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188465    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/390448c5-9242-4562-84e5-0307125ba1ad-etc-tuned\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.190481 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188486    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-host-run-k8s-cni-cncf-io\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.190481 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188490    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1fe146d4-3a94-48d2-b383-ea0a4c301b54-cnibin\") pod \"multus-additional-cni-plugins-7j9mn\" (UID: \"1fe146d4-3a94-48d2-b383-ea0a4c301b54\") " pod="openshift-multus/multus-additional-cni-plugins-7j9mn"
Sep 11 00:58:06.190481 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188511    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-host-var-lib-cni-bin\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.190481 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188515    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ff3ee2a6-e101-4cde-8d89-489f1db81ab8-dbus\") pod \"global-pull-secret-syncer-jbpsj\" (UID: \"ff3ee2a6-e101-4cde-8d89-489f1db81ab8\") " pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:58:06.190481 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188518    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d097ee33-8900-43b7-9eae-35ac88373d1d-hosts-file\") pod \"node-resolver-zw22q\" (UID: \"d097ee33-8900-43b7-9eae-35ac88373d1d\") " pod="openshift-dns/node-resolver-zw22q"
Sep 11 00:58:06.190481 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188536    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-etc-openvswitch\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.190481 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188563    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66bcw\" (UniqueName: \"kubernetes.io/projected/1625cd1f-5711-4d05-9cfc-74a2a2c1c637-kube-api-access-66bcw\") pod \"node-ca-hh5wm\" (UID: \"1625cd1f-5711-4d05-9cfc-74a2a2c1c637\") " pod="openshift-image-registry/node-ca-hh5wm"
Sep 11 00:58:06.190481 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188569    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-host-var-lib-cni-bin\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.190481 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188568    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-host-run-k8s-cni-cncf-io\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.190481 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188584    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/390448c5-9242-4562-84e5-0307125ba1ad-sys\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.191273 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188605    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-etc-openvswitch\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.191273 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188606    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8ad98990-55b3-4499-9b32-a8b394175160-etc-selinux\") pod \"aws-ebs-csi-driver-node-zhmsk\" (UID: \"8ad98990-55b3-4499-9b32-a8b394175160\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhmsk"
Sep 11 00:58:06.191273 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188641    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-system-cni-dir\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.191273 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188678    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8ad98990-55b3-4499-9b32-a8b394175160-etc-selinux\") pod \"aws-ebs-csi-driver-node-zhmsk\" (UID: \"8ad98990-55b3-4499-9b32-a8b394175160\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhmsk"
Sep 11 00:58:06.191273 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188691    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-system-cni-dir\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.191273 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188693    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/390448c5-9242-4562-84e5-0307125ba1ad-sys\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.191273 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188728    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-cni-binary-copy\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.191273 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188752    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-node-log\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.191273 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188770    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5176ce4f-e447-4f42-a338-2f1ee4761210-ovnkube-config\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.191273 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188792    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5176ce4f-e447-4f42-a338-2f1ee4761210-ovnkube-script-lib\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.191273 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188805    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-node-log\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.191273 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188815    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1625cd1f-5711-4d05-9cfc-74a2a2c1c637-serviceca\") pod \"node-ca-hh5wm\" (UID: \"1625cd1f-5711-4d05-9cfc-74a2a2c1c637\") " pod="openshift-image-registry/node-ca-hh5wm"
Sep 11 00:58:06.191273 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188839    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff3ee2a6-e101-4cde-8d89-489f1db81ab8-original-pull-secret\") pod \"global-pull-secret-syncer-jbpsj\" (UID: \"ff3ee2a6-e101-4cde-8d89-489f1db81ab8\") " pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:58:06.191273 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188862    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8ad98990-55b3-4499-9b32-a8b394175160-socket-dir\") pod \"aws-ebs-csi-driver-node-zhmsk\" (UID: \"8ad98990-55b3-4499-9b32-a8b394175160\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhmsk"
Sep 11 00:58:06.191273 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188886    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/390448c5-9242-4562-84e5-0307125ba1ad-etc-sysctl-d\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.191273 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188925    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-host-cni-netd\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.191273 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188947    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5176ce4f-e447-4f42-a338-2f1ee4761210-ovn-node-metrics-cert\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.191273 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188966    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/390448c5-9242-4562-84e5-0307125ba1ad-etc-modprobe-d\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.191765 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188990    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/396a7cf9-e334-407d-ad45-789fb8dfba05-konnectivity-ca\") pod \"konnectivity-agent-l5nhz\" (UID: \"396a7cf9-e334-407d-ad45-789fb8dfba05\") " pod="kube-system/konnectivity-agent-l5nhz"
Sep 11 00:58:06.191765 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.189016    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ff3ee2a6-e101-4cde-8d89-489f1db81ab8-kubelet-config\") pod \"global-pull-secret-syncer-jbpsj\" (UID: \"ff3ee2a6-e101-4cde-8d89-489f1db81ab8\") " pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:58:06.191765 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.189040    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-multus-cni-dir\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.191765 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.189044    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/390448c5-9242-4562-84e5-0307125ba1ad-etc-sysctl-d\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.191765 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.189062    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-host-run-multus-certs\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.191765 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.189086    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-var-lib-openvswitch\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.191765 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.189109    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/396a7cf9-e334-407d-ad45-789fb8dfba05-agent-certs\") pod \"konnectivity-agent-l5nhz\" (UID: \"396a7cf9-e334-407d-ad45-789fb8dfba05\") " pod="kube-system/konnectivity-agent-l5nhz"
Sep 11 00:58:06.191765 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.189133    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77093864-53b6-4044-952a-d8f1a965dd1d-metrics-certs\") pod \"network-metrics-daemon-9s5p9\" (UID: \"77093864-53b6-4044-952a-d8f1a965dd1d\") " pod="openshift-multus/network-metrics-daemon-9s5p9"
Sep 11 00:58:06.191765 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.189156    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-host-slash\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.191765 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.189178    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-run-openvswitch\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.191765 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.189204    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tdws\" (UniqueName: \"kubernetes.io/projected/526304d7-8bd3-4196-a695-f5845e8846a1-kube-api-access-2tdws\") pod \"network-check-target-9hl4x\" (UID: \"526304d7-8bd3-4196-a695-f5845e8846a1\") " pod="openshift-network-diagnostics/network-check-target-9hl4x"
Sep 11 00:58:06.191765 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.189225    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1fe146d4-3a94-48d2-b383-ea0a4c301b54-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7j9mn\" (UID: \"1fe146d4-3a94-48d2-b383-ea0a4c301b54\") " pod="openshift-multus/multus-additional-cni-plugins-7j9mn"
Sep 11 00:58:06.191765 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.189247    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/390448c5-9242-4562-84e5-0307125ba1ad-etc-kubernetes\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.191765 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.189270    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/390448c5-9242-4562-84e5-0307125ba1ad-etc-sysctl-conf\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.191765 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.189292    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/390448c5-9242-4562-84e5-0307125ba1ad-run\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.191765 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.189318    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5176ce4f-e447-4f42-a338-2f1ee4761210-env-overrides\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.191765 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.189336    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5176ce4f-e447-4f42-a338-2f1ee4761210-ovnkube-config\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.192234 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.189343    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh9kz\" (UniqueName: \"kubernetes.io/projected/c29dcdd7-781d-411d-b6e1-4cb59de9be56-kube-api-access-wh9kz\") pod \"iptables-alerter-qx27q\" (UID: \"c29dcdd7-781d-411d-b6e1-4cb59de9be56\") " pod="openshift-network-operator/iptables-alerter-qx27q"
Sep 11 00:58:06.192234 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.189367    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1fe146d4-3a94-48d2-b383-ea0a4c301b54-system-cni-dir\") pod \"multus-additional-cni-plugins-7j9mn\" (UID: \"1fe146d4-3a94-48d2-b383-ea0a4c301b54\") " pod="openshift-multus/multus-additional-cni-plugins-7j9mn"
Sep 11 00:58:06.192234 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.189413    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-cni-binary-copy\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.192234 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:06.189425    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 00:58:06.192234 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.189442    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5176ce4f-e447-4f42-a338-2f1ee4761210-ovnkube-script-lib\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.192234 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.189447    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1625cd1f-5711-4d05-9cfc-74a2a2c1c637-serviceca\") pod \"node-ca-hh5wm\" (UID: \"1625cd1f-5711-4d05-9cfc-74a2a2c1c637\") " pod="openshift-image-registry/node-ca-hh5wm"
Sep 11 00:58:06.192234 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.189492    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-host-cni-netd\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.192234 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:06.189516    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff3ee2a6-e101-4cde-8d89-489f1db81ab8-original-pull-secret podName:ff3ee2a6-e101-4cde-8d89-489f1db81ab8 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:06.689472408 +0000 UTC m=+2.086664562 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ff3ee2a6-e101-4cde-8d89-489f1db81ab8-original-pull-secret") pod "global-pull-secret-syncer-jbpsj" (UID: "ff3ee2a6-e101-4cde-8d89-489f1db81ab8") : object "kube-system"/"original-pull-secret" not registered
Sep 11 00:58:06.192234 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.189513    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1fe146d4-3a94-48d2-b383-ea0a4c301b54-system-cni-dir\") pod \"multus-additional-cni-plugins-7j9mn\" (UID: \"1fe146d4-3a94-48d2-b383-ea0a4c301b54\") " pod="openshift-multus/multus-additional-cni-plugins-7j9mn"
Sep 11 00:58:06.192234 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.189533    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-host-slash\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.192234 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.189540    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8ad98990-55b3-4499-9b32-a8b394175160-socket-dir\") pod \"aws-ebs-csi-driver-node-zhmsk\" (UID: \"8ad98990-55b3-4499-9b32-a8b394175160\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhmsk"
Sep 11 00:58:06.192234 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.188966    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-multus-daemon-config\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.192234 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.189573    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-run-openvswitch\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.192234 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.189586    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ff3ee2a6-e101-4cde-8d89-489f1db81ab8-kubelet-config\") pod \"global-pull-secret-syncer-jbpsj\" (UID: \"ff3ee2a6-e101-4cde-8d89-489f1db81ab8\") " pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:58:06.192234 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.189643    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/390448c5-9242-4562-84e5-0307125ba1ad-etc-modprobe-d\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.192234 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:06.189650    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 00:58:06.192234 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.189691    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-host-run-multus-certs\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.192721 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:06.189710    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77093864-53b6-4044-952a-d8f1a965dd1d-metrics-certs podName:77093864-53b6-4044-952a-d8f1a965dd1d nodeName:}" failed. No retries permitted until 2026-09-11 00:58:06.689696204 +0000 UTC m=+2.086888361 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/77093864-53b6-4044-952a-d8f1a965dd1d-metrics-certs") pod "network-metrics-daemon-9s5p9" (UID: "77093864-53b6-4044-952a-d8f1a965dd1d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 00:58:06.192721 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.189769    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/390448c5-9242-4562-84e5-0307125ba1ad-run\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.192721 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.189821    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/390448c5-9242-4562-84e5-0307125ba1ad-etc-kubernetes\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.192721 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.189903    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-multus-cni-dir\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.192721 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.189948    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/390448c5-9242-4562-84e5-0307125ba1ad-etc-sysctl-conf\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.192721 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.189976    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5176ce4f-e447-4f42-a338-2f1ee4761210-var-lib-openvswitch\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.192721 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.190037    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/396a7cf9-e334-407d-ad45-789fb8dfba05-konnectivity-ca\") pod \"konnectivity-agent-l5nhz\" (UID: \"396a7cf9-e334-407d-ad45-789fb8dfba05\") " pod="kube-system/konnectivity-agent-l5nhz"
Sep 11 00:58:06.192721 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.190113    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1fe146d4-3a94-48d2-b383-ea0a4c301b54-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7j9mn\" (UID: \"1fe146d4-3a94-48d2-b383-ea0a4c301b54\") " pod="openshift-multus/multus-additional-cni-plugins-7j9mn"
Sep 11 00:58:06.192721 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.190300    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5176ce4f-e447-4f42-a338-2f1ee4761210-env-overrides\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.192721 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.191396    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/390448c5-9242-4562-84e5-0307125ba1ad-etc-tuned\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.192721 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.191430    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/390448c5-9242-4562-84e5-0307125ba1ad-tmp\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.192721 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.191720    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5176ce4f-e447-4f42-a338-2f1ee4761210-ovn-node-metrics-cert\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.192721 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.192175    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/396a7cf9-e334-407d-ad45-789fb8dfba05-agent-certs\") pod \"konnectivity-agent-l5nhz\" (UID: \"396a7cf9-e334-407d-ad45-789fb8dfba05\") " pod="kube-system/konnectivity-agent-l5nhz"
Sep 11 00:58:06.196447 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:06.196088    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 00:58:06.196447 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:06.196113    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 00:58:06.196447 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:06.196124    2574 projected.go:196] Error preparing data for projected volume kube-api-access-2tdws for pod openshift-network-diagnostics/network-check-target-9hl4x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 00:58:06.196447 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:06.196243    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/526304d7-8bd3-4196-a695-f5845e8846a1-kube-api-access-2tdws podName:526304d7-8bd3-4196-a695-f5845e8846a1 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:06.696228542 +0000 UTC m=+2.093420694 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2tdws" (UniqueName: "kubernetes.io/projected/526304d7-8bd3-4196-a695-f5845e8846a1-kube-api-access-2tdws") pod "network-check-target-9hl4x" (UID: "526304d7-8bd3-4196-a695-f5845e8846a1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 00:58:06.196447 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.196389    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8lzw\" (UniqueName: \"kubernetes.io/projected/f4363dbb-d8eb-4eb6-af98-f3327b90c84c-kube-api-access-h8lzw\") pod \"multus-w9q8q\" (UID: \"f4363dbb-d8eb-4eb6-af98-f3327b90c84c\") " pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.196751 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.196584    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xkwk\" (UniqueName: \"kubernetes.io/projected/d097ee33-8900-43b7-9eae-35ac88373d1d-kube-api-access-4xkwk\") pod \"node-resolver-zw22q\" (UID: \"d097ee33-8900-43b7-9eae-35ac88373d1d\") " pod="openshift-dns/node-resolver-zw22q"
Sep 11 00:58:06.197317 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.197112    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trqzg\" (UniqueName: \"kubernetes.io/projected/77093864-53b6-4044-952a-d8f1a965dd1d-kube-api-access-trqzg\") pod \"network-metrics-daemon-9s5p9\" (UID: \"77093864-53b6-4044-952a-d8f1a965dd1d\") " pod="openshift-multus/network-metrics-daemon-9s5p9"
Sep 11 00:58:06.197317 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.197122    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glssd\" (UniqueName: \"kubernetes.io/projected/8ad98990-55b3-4499-9b32-a8b394175160-kube-api-access-glssd\") pod \"aws-ebs-csi-driver-node-zhmsk\" (UID: \"8ad98990-55b3-4499-9b32-a8b394175160\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhmsk"
Sep 11 00:58:06.197317 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.197278    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mp5x4\" (UniqueName: \"kubernetes.io/projected/1fe146d4-3a94-48d2-b383-ea0a4c301b54-kube-api-access-mp5x4\") pod \"multus-additional-cni-plugins-7j9mn\" (UID: \"1fe146d4-3a94-48d2-b383-ea0a4c301b54\") " pod="openshift-multus/multus-additional-cni-plugins-7j9mn"
Sep 11 00:58:06.197497 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.197395    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66bcw\" (UniqueName: \"kubernetes.io/projected/1625cd1f-5711-4d05-9cfc-74a2a2c1c637-kube-api-access-66bcw\") pod \"node-ca-hh5wm\" (UID: \"1625cd1f-5711-4d05-9cfc-74a2a2c1c637\") " pod="openshift-image-registry/node-ca-hh5wm"
Sep 11 00:58:06.197826 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.197806    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5npk\" (UniqueName: \"kubernetes.io/projected/5176ce4f-e447-4f42-a338-2f1ee4761210-kube-api-access-f5npk\") pod \"ovnkube-node-d7crj\" (UID: \"5176ce4f-e447-4f42-a338-2f1ee4761210\") " pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.198187 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.198166    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdndj\" (UniqueName: \"kubernetes.io/projected/390448c5-9242-4562-84e5-0307125ba1ad-kube-api-access-gdndj\") pod \"tuned-8q8bw\" (UID: \"390448c5-9242-4562-84e5-0307125ba1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.198767 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.198752    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wh9kz\" (UniqueName: \"kubernetes.io/projected/c29dcdd7-781d-411d-b6e1-4cb59de9be56-kube-api-access-wh9kz\") pod \"iptables-alerter-qx27q\" (UID: \"c29dcdd7-781d-411d-b6e1-4cb59de9be56\") " pod="openshift-network-operator/iptables-alerter-qx27q"
Sep 11 00:58:06.300231 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.300186    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:06.310220 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:06.310204    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5176ce4f_e447_4f42_a338_2f1ee4761210.slice/crio-cedbb13e183807b31cf899e62ad211b740d61045d45e610ff7eaba31afc01b54 WatchSource:0}: Error finding container cedbb13e183807b31cf899e62ad211b740d61045d45e610ff7eaba31afc01b54: Status 404 returned error can't find the container with id cedbb13e183807b31cf899e62ad211b740d61045d45e610ff7eaba31afc01b54
Sep 11 00:58:06.311748 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.311734    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7j9mn"
Sep 11 00:58:06.319656 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:06.319638    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fe146d4_3a94_48d2_b383_ea0a4c301b54.slice/crio-fe8e6c04336bdea5ba547dcc96a323ba685be2562dc467e113ed8207c4415d04 WatchSource:0}: Error finding container fe8e6c04336bdea5ba547dcc96a323ba685be2562dc467e113ed8207c4415d04: Status 404 returned error can't find the container with id fe8e6c04336bdea5ba547dcc96a323ba685be2562dc467e113ed8207c4415d04
Sep 11 00:58:06.327456 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.327439    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8q8bw"
Sep 11 00:58:06.335369 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:06.335351    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod390448c5_9242_4562_84e5_0307125ba1ad.slice/crio-9d6d34ea2c9b54d309e45fd653a23eb1de69abf8aa6d26f6bf9855b462a70649 WatchSource:0}: Error finding container 9d6d34ea2c9b54d309e45fd653a23eb1de69abf8aa6d26f6bf9855b462a70649: Status 404 returned error can't find the container with id 9d6d34ea2c9b54d309e45fd653a23eb1de69abf8aa6d26f6bf9855b462a70649
Sep 11 00:58:06.344052 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.344038    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhmsk"
Sep 11 00:58:06.351445 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:06.351429    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ad98990_55b3_4499_9b32_a8b394175160.slice/crio-895ada95d8b731e8059c0cb8aaaa6e4396b31d3f2a4b679bb7753ebbde01c204 WatchSource:0}: Error finding container 895ada95d8b731e8059c0cb8aaaa6e4396b31d3f2a4b679bb7753ebbde01c204: Status 404 returned error can't find the container with id 895ada95d8b731e8059c0cb8aaaa6e4396b31d3f2a4b679bb7753ebbde01c204
Sep 11 00:58:06.353735 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.353722    2574 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 00:58:06.362500 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.362485    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qx27q"
Sep 11 00:58:06.370158 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:06.370140    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc29dcdd7_781d_411d_b6e1_4cb59de9be56.slice/crio-55103021a4fbe3e41f30c33917ea7f829e66a9e29c20979db3e6183d866c747f WatchSource:0}: Error finding container 55103021a4fbe3e41f30c33917ea7f829e66a9e29c20979db3e6183d866c747f: Status 404 returned error can't find the container with id 55103021a4fbe3e41f30c33917ea7f829e66a9e29c20979db3e6183d866c747f
Sep 11 00:58:06.382038 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.382024    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zw22q"
Sep 11 00:58:06.387818 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.387803    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l5nhz"
Sep 11 00:58:06.390487 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:06.390469    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd097ee33_8900_43b7_9eae_35ac88373d1d.slice/crio-a4b5af0a7fcbeee010f2d8150428d39cfb4e985328b394abd9ce3b9e549a1264 WatchSource:0}: Error finding container a4b5af0a7fcbeee010f2d8150428d39cfb4e985328b394abd9ce3b9e549a1264: Status 404 returned error can't find the container with id a4b5af0a7fcbeee010f2d8150428d39cfb4e985328b394abd9ce3b9e549a1264
Sep 11 00:58:06.395300 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:06.395283    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod396a7cf9_e334_407d_ad45_789fb8dfba05.slice/crio-2d4e22b4f15b9c3dcedfd710e4cd95516ee3021d5be492389df62da87dede6ca WatchSource:0}: Error finding container 2d4e22b4f15b9c3dcedfd710e4cd95516ee3021d5be492389df62da87dede6ca: Status 404 returned error can't find the container with id 2d4e22b4f15b9c3dcedfd710e4cd95516ee3021d5be492389df62da87dede6ca
Sep 11 00:58:06.400394 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.400380    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hh5wm"
Sep 11 00:58:06.404973 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.404952    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-w9q8q"
Sep 11 00:58:06.408820 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:06.408803    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1625cd1f_5711_4d05_9cfc_74a2a2c1c637.slice/crio-738e8c1aad176cbaf61738d984b594625761c9edb11c32ff102a9c0b4e8cdf90 WatchSource:0}: Error finding container 738e8c1aad176cbaf61738d984b594625761c9edb11c32ff102a9c0b4e8cdf90: Status 404 returned error can't find the container with id 738e8c1aad176cbaf61738d984b594625761c9edb11c32ff102a9c0b4e8cdf90
Sep 11 00:58:06.413223 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:06.413205    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4363dbb_d8eb_4eb6_af98_f3327b90c84c.slice/crio-bb551887ea5d34d0e4fb4677ae557c0591aac9d4a40153e2c42a6fd99e4a5a59 WatchSource:0}: Error finding container bb551887ea5d34d0e4fb4677ae557c0591aac9d4a40153e2c42a6fd99e4a5a59: Status 404 returned error can't find the container with id bb551887ea5d34d0e4fb4677ae557c0591aac9d4a40153e2c42a6fd99e4a5a59
Sep 11 00:58:06.693053 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.693018    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff3ee2a6-e101-4cde-8d89-489f1db81ab8-original-pull-secret\") pod \"global-pull-secret-syncer-jbpsj\" (UID: \"ff3ee2a6-e101-4cde-8d89-489f1db81ab8\") " pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:58:06.693249 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.693077    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77093864-53b6-4044-952a-d8f1a965dd1d-metrics-certs\") pod \"network-metrics-daemon-9s5p9\" (UID: \"77093864-53b6-4044-952a-d8f1a965dd1d\") " pod="openshift-multus/network-metrics-daemon-9s5p9"
Sep 11 00:58:06.693340 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:06.693250    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 00:58:06.693340 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:06.693312    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77093864-53b6-4044-952a-d8f1a965dd1d-metrics-certs podName:77093864-53b6-4044-952a-d8f1a965dd1d nodeName:}" failed. No retries permitted until 2026-09-11 00:58:07.693290102 +0000 UTC m=+3.090482271 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/77093864-53b6-4044-952a-d8f1a965dd1d-metrics-certs") pod "network-metrics-daemon-9s5p9" (UID: "77093864-53b6-4044-952a-d8f1a965dd1d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 00:58:06.693785 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:06.693750    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 00:58:06.693900 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:06.693827    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff3ee2a6-e101-4cde-8d89-489f1db81ab8-original-pull-secret podName:ff3ee2a6-e101-4cde-8d89-489f1db81ab8 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:07.693809693 +0000 UTC m=+3.091001845 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ff3ee2a6-e101-4cde-8d89-489f1db81ab8-original-pull-secret") pod "global-pull-secret-syncer-jbpsj" (UID: "ff3ee2a6-e101-4cde-8d89-489f1db81ab8") : object "kube-system"/"original-pull-secret" not registered
Sep 11 00:58:06.794095 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:06.794058    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tdws\" (UniqueName: \"kubernetes.io/projected/526304d7-8bd3-4196-a695-f5845e8846a1-kube-api-access-2tdws\") pod \"network-check-target-9hl4x\" (UID: \"526304d7-8bd3-4196-a695-f5845e8846a1\") " pod="openshift-network-diagnostics/network-check-target-9hl4x"
Sep 11 00:58:06.794273 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:06.794213    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 00:58:06.794273 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:06.794232    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 00:58:06.794273 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:06.794246    2574 projected.go:196] Error preparing data for projected volume kube-api-access-2tdws for pod openshift-network-diagnostics/network-check-target-9hl4x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 00:58:06.794556 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:06.794326    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/526304d7-8bd3-4196-a695-f5845e8846a1-kube-api-access-2tdws podName:526304d7-8bd3-4196-a695-f5845e8846a1 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:07.794306065 +0000 UTC m=+3.191498238 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2tdws" (UniqueName: "kubernetes.io/projected/526304d7-8bd3-4196-a695-f5845e8846a1-kube-api-access-2tdws") pod "network-check-target-9hl4x" (UID: "526304d7-8bd3-4196-a695-f5845e8846a1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 00:58:07.024849 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:07.024741    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 00:53:06 +0000 UTC" deadline="2028-04-27 15:04:37.637868063 +0000 UTC"
Sep 11 00:58:07.024849 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:07.024801    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14270h6m30.613071336s"
Sep 11 00:58:07.070127 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:07.070099    2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 00:58:07.128350 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:07.128281    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zw22q" event={"ID":"d097ee33-8900-43b7-9eae-35ac88373d1d","Type":"ContainerStarted","Data":"a4b5af0a7fcbeee010f2d8150428d39cfb4e985328b394abd9ce3b9e549a1264"}
Sep 11 00:58:07.144422 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:07.144385    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d7crj" event={"ID":"5176ce4f-e447-4f42-a338-2f1ee4761210","Type":"ContainerStarted","Data":"cedbb13e183807b31cf899e62ad211b740d61045d45e610ff7eaba31afc01b54"}
Sep 11 00:58:07.170341 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:07.170273    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-235.ec2.internal" event={"ID":"9f1f5adc45e045b22e89459332dfb6a0","Type":"ContainerStarted","Data":"fe75795b91ed4bc4e6a78de66736b62e4dbec5e3227d3f8bdcbbc7fd50eaecf1"}
Sep 11 00:58:07.189223 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:07.189188    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-235.ec2.internal" event={"ID":"d958aa85aad010b15ac136386e20db34","Type":"ContainerStarted","Data":"ab9b61f53be03f21cc566b10b78f85b15e04946c32a067b34f36b2ffff035448"}
Sep 11 00:58:07.206543 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:07.206510    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-w9q8q" event={"ID":"f4363dbb-d8eb-4eb6-af98-f3327b90c84c","Type":"ContainerStarted","Data":"bb551887ea5d34d0e4fb4677ae557c0591aac9d4a40153e2c42a6fd99e4a5a59"}
Sep 11 00:58:07.223433 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:07.219245    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hh5wm" event={"ID":"1625cd1f-5711-4d05-9cfc-74a2a2c1c637","Type":"ContainerStarted","Data":"738e8c1aad176cbaf61738d984b594625761c9edb11c32ff102a9c0b4e8cdf90"}
Sep 11 00:58:07.241345 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:07.241303    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l5nhz" event={"ID":"396a7cf9-e334-407d-ad45-789fb8dfba05","Type":"ContainerStarted","Data":"2d4e22b4f15b9c3dcedfd710e4cd95516ee3021d5be492389df62da87dede6ca"}
Sep 11 00:58:07.272307 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:07.272212    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qx27q" event={"ID":"c29dcdd7-781d-411d-b6e1-4cb59de9be56","Type":"ContainerStarted","Data":"55103021a4fbe3e41f30c33917ea7f829e66a9e29c20979db3e6183d866c747f"}
Sep 11 00:58:07.293009 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:07.292904    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhmsk" event={"ID":"8ad98990-55b3-4499-9b32-a8b394175160","Type":"ContainerStarted","Data":"895ada95d8b731e8059c0cb8aaaa6e4396b31d3f2a4b679bb7753ebbde01c204"}
Sep 11 00:58:07.299769 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:07.299743    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8q8bw" event={"ID":"390448c5-9242-4562-84e5-0307125ba1ad","Type":"ContainerStarted","Data":"9d6d34ea2c9b54d309e45fd653a23eb1de69abf8aa6d26f6bf9855b462a70649"}
Sep 11 00:58:07.316342 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:07.316309    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7j9mn" event={"ID":"1fe146d4-3a94-48d2-b383-ea0a4c301b54","Type":"ContainerStarted","Data":"fe8e6c04336bdea5ba547dcc96a323ba685be2562dc467e113ed8207c4415d04"}
Sep 11 00:58:07.515927 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:07.515708    2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 00:58:07.708491 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:07.708453    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff3ee2a6-e101-4cde-8d89-489f1db81ab8-original-pull-secret\") pod \"global-pull-secret-syncer-jbpsj\" (UID: \"ff3ee2a6-e101-4cde-8d89-489f1db81ab8\") " pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:58:07.708685 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:07.708509    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77093864-53b6-4044-952a-d8f1a965dd1d-metrics-certs\") pod \"network-metrics-daemon-9s5p9\" (UID: \"77093864-53b6-4044-952a-d8f1a965dd1d\") " pod="openshift-multus/network-metrics-daemon-9s5p9"
Sep 11 00:58:07.708685 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:07.708663    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 00:58:07.708802 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:07.708722    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77093864-53b6-4044-952a-d8f1a965dd1d-metrics-certs podName:77093864-53b6-4044-952a-d8f1a965dd1d nodeName:}" failed. No retries permitted until 2026-09-11 00:58:09.708701321 +0000 UTC m=+5.105893490 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/77093864-53b6-4044-952a-d8f1a965dd1d-metrics-certs") pod "network-metrics-daemon-9s5p9" (UID: "77093864-53b6-4044-952a-d8f1a965dd1d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 00:58:07.709191 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:07.709169    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 00:58:07.709305 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:07.709236    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff3ee2a6-e101-4cde-8d89-489f1db81ab8-original-pull-secret podName:ff3ee2a6-e101-4cde-8d89-489f1db81ab8 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:09.709220424 +0000 UTC m=+5.106412592 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ff3ee2a6-e101-4cde-8d89-489f1db81ab8-original-pull-secret") pod "global-pull-secret-syncer-jbpsj" (UID: "ff3ee2a6-e101-4cde-8d89-489f1db81ab8") : object "kube-system"/"original-pull-secret" not registered
Sep 11 00:58:07.809365 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:07.809322    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tdws\" (UniqueName: \"kubernetes.io/projected/526304d7-8bd3-4196-a695-f5845e8846a1-kube-api-access-2tdws\") pod \"network-check-target-9hl4x\" (UID: \"526304d7-8bd3-4196-a695-f5845e8846a1\") " pod="openshift-network-diagnostics/network-check-target-9hl4x"
Sep 11 00:58:07.809561 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:07.809513    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 00:58:07.809561 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:07.809538    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 00:58:07.809561 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:07.809553    2574 projected.go:196] Error preparing data for projected volume kube-api-access-2tdws for pod openshift-network-diagnostics/network-check-target-9hl4x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 00:58:07.809728 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:07.809612    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/526304d7-8bd3-4196-a695-f5845e8846a1-kube-api-access-2tdws podName:526304d7-8bd3-4196-a695-f5845e8846a1 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:09.80959334 +0000 UTC m=+5.206785508 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2tdws" (UniqueName: "kubernetes.io/projected/526304d7-8bd3-4196-a695-f5845e8846a1-kube-api-access-2tdws") pod "network-check-target-9hl4x" (UID: "526304d7-8bd3-4196-a695-f5845e8846a1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 00:58:08.025235 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:08.025117    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 00:53:06 +0000 UTC" deadline="2028-04-02 20:53:33.224864093 +0000 UTC"
Sep 11 00:58:08.025235 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:08.025181    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13675h55m25.199687093s"
Sep 11 00:58:08.096008 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:08.094529    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5p9"
Sep 11 00:58:08.096008 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:08.094683    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9s5p9" podUID="77093864-53b6-4044-952a-d8f1a965dd1d"
Sep 11 00:58:08.096008 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:08.095133    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hl4x"
Sep 11 00:58:08.096008 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:08.095225    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hl4x" podUID="526304d7-8bd3-4196-a695-f5845e8846a1"
Sep 11 00:58:08.096008 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:08.095292    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:58:08.096008 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:08.095365    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbpsj" podUID="ff3ee2a6-e101-4cde-8d89-489f1db81ab8"
Sep 11 00:58:08.372958 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:08.372618    2574 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 00:58:09.729077 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:09.729038    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff3ee2a6-e101-4cde-8d89-489f1db81ab8-original-pull-secret\") pod \"global-pull-secret-syncer-jbpsj\" (UID: \"ff3ee2a6-e101-4cde-8d89-489f1db81ab8\") " pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:58:09.729597 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:09.729103    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77093864-53b6-4044-952a-d8f1a965dd1d-metrics-certs\") pod \"network-metrics-daemon-9s5p9\" (UID: \"77093864-53b6-4044-952a-d8f1a965dd1d\") " pod="openshift-multus/network-metrics-daemon-9s5p9"
Sep 11 00:58:09.729597 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:09.729227    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 00:58:09.729597 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:09.729247    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 00:58:09.729597 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:09.729322    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff3ee2a6-e101-4cde-8d89-489f1db81ab8-original-pull-secret podName:ff3ee2a6-e101-4cde-8d89-489f1db81ab8 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:13.729299898 +0000 UTC m=+9.126492067 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ff3ee2a6-e101-4cde-8d89-489f1db81ab8-original-pull-secret") pod "global-pull-secret-syncer-jbpsj" (UID: "ff3ee2a6-e101-4cde-8d89-489f1db81ab8") : object "kube-system"/"original-pull-secret" not registered
Sep 11 00:58:09.729597 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:09.729346    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77093864-53b6-4044-952a-d8f1a965dd1d-metrics-certs podName:77093864-53b6-4044-952a-d8f1a965dd1d nodeName:}" failed. No retries permitted until 2026-09-11 00:58:13.729334666 +0000 UTC m=+9.126526822 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/77093864-53b6-4044-952a-d8f1a965dd1d-metrics-certs") pod "network-metrics-daemon-9s5p9" (UID: "77093864-53b6-4044-952a-d8f1a965dd1d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 00:58:09.830218 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:09.830174    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tdws\" (UniqueName: \"kubernetes.io/projected/526304d7-8bd3-4196-a695-f5845e8846a1-kube-api-access-2tdws\") pod \"network-check-target-9hl4x\" (UID: \"526304d7-8bd3-4196-a695-f5845e8846a1\") " pod="openshift-network-diagnostics/network-check-target-9hl4x"
Sep 11 00:58:09.830410 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:09.830384    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 00:58:09.830410 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:09.830407    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 00:58:09.830531 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:09.830423    2574 projected.go:196] Error preparing data for projected volume kube-api-access-2tdws for pod openshift-network-diagnostics/network-check-target-9hl4x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 00:58:09.830531 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:09.830520    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/526304d7-8bd3-4196-a695-f5845e8846a1-kube-api-access-2tdws podName:526304d7-8bd3-4196-a695-f5845e8846a1 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:13.830499864 +0000 UTC m=+9.227692032 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2tdws" (UniqueName: "kubernetes.io/projected/526304d7-8bd3-4196-a695-f5845e8846a1-kube-api-access-2tdws") pod "network-check-target-9hl4x" (UID: "526304d7-8bd3-4196-a695-f5845e8846a1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 00:58:10.094438 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:10.094366    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:58:10.094597 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:10.094493    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbpsj" podUID="ff3ee2a6-e101-4cde-8d89-489f1db81ab8"
Sep 11 00:58:10.094957 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:10.094931    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5p9"
Sep 11 00:58:10.095110 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:10.095082    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9s5p9" podUID="77093864-53b6-4044-952a-d8f1a965dd1d"
Sep 11 00:58:10.095175 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:10.095138    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hl4x"
Sep 11 00:58:10.095235 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:10.095215    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hl4x" podUID="526304d7-8bd3-4196-a695-f5845e8846a1"
Sep 11 00:58:12.094165 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:12.094129    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:58:12.094632 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:12.094268    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbpsj" podUID="ff3ee2a6-e101-4cde-8d89-489f1db81ab8"
Sep 11 00:58:12.094632 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:12.094338    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5p9"
Sep 11 00:58:12.094632 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:12.094427    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9s5p9" podUID="77093864-53b6-4044-952a-d8f1a965dd1d"
Sep 11 00:58:12.094632 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:12.094470    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hl4x"
Sep 11 00:58:12.094632 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:12.094536    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hl4x" podUID="526304d7-8bd3-4196-a695-f5845e8846a1"
Sep 11 00:58:13.765897 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:13.765638    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff3ee2a6-e101-4cde-8d89-489f1db81ab8-original-pull-secret\") pod \"global-pull-secret-syncer-jbpsj\" (UID: \"ff3ee2a6-e101-4cde-8d89-489f1db81ab8\") " pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:58:13.765897 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:13.765696    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77093864-53b6-4044-952a-d8f1a965dd1d-metrics-certs\") pod \"network-metrics-daemon-9s5p9\" (UID: \"77093864-53b6-4044-952a-d8f1a965dd1d\") " pod="openshift-multus/network-metrics-daemon-9s5p9"
Sep 11 00:58:13.765897 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:13.765813    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 00:58:13.765897 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:13.765837    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 00:58:13.765897 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:13.765871    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77093864-53b6-4044-952a-d8f1a965dd1d-metrics-certs podName:77093864-53b6-4044-952a-d8f1a965dd1d nodeName:}" failed. No retries permitted until 2026-09-11 00:58:21.765852906 +0000 UTC m=+17.163045073 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/77093864-53b6-4044-952a-d8f1a965dd1d-metrics-certs") pod "network-metrics-daemon-9s5p9" (UID: "77093864-53b6-4044-952a-d8f1a965dd1d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 00:58:13.765897 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:13.765901    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff3ee2a6-e101-4cde-8d89-489f1db81ab8-original-pull-secret podName:ff3ee2a6-e101-4cde-8d89-489f1db81ab8 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:21.765882689 +0000 UTC m=+17.163074847 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ff3ee2a6-e101-4cde-8d89-489f1db81ab8-original-pull-secret") pod "global-pull-secret-syncer-jbpsj" (UID: "ff3ee2a6-e101-4cde-8d89-489f1db81ab8") : object "kube-system"/"original-pull-secret" not registered
Sep 11 00:58:13.866419 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:13.866332    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tdws\" (UniqueName: \"kubernetes.io/projected/526304d7-8bd3-4196-a695-f5845e8846a1-kube-api-access-2tdws\") pod \"network-check-target-9hl4x\" (UID: \"526304d7-8bd3-4196-a695-f5845e8846a1\") " pod="openshift-network-diagnostics/network-check-target-9hl4x"
Sep 11 00:58:13.866597 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:13.866507    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 00:58:13.866597 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:13.866531    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 00:58:13.866597 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:13.866544    2574 projected.go:196] Error preparing data for projected volume kube-api-access-2tdws for pod openshift-network-diagnostics/network-check-target-9hl4x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 00:58:13.866727 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:13.866603    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/526304d7-8bd3-4196-a695-f5845e8846a1-kube-api-access-2tdws podName:526304d7-8bd3-4196-a695-f5845e8846a1 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:21.866584477 +0000 UTC m=+17.263776632 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2tdws" (UniqueName: "kubernetes.io/projected/526304d7-8bd3-4196-a695-f5845e8846a1-kube-api-access-2tdws") pod "network-check-target-9hl4x" (UID: "526304d7-8bd3-4196-a695-f5845e8846a1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 00:58:14.094836 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:14.094400    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:58:14.094836 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:14.094439    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5p9"
Sep 11 00:58:14.094836 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:14.094531    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbpsj" podUID="ff3ee2a6-e101-4cde-8d89-489f1db81ab8"
Sep 11 00:58:14.094836 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:14.094607    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9s5p9" podUID="77093864-53b6-4044-952a-d8f1a965dd1d"
Sep 11 00:58:14.094836 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:14.094643    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hl4x"
Sep 11 00:58:14.094836 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:14.094732    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hl4x" podUID="526304d7-8bd3-4196-a695-f5845e8846a1"
Sep 11 00:58:16.094072 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:16.094039    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hl4x"
Sep 11 00:58:16.094532 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:16.094081    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:58:16.094532 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:16.094089    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5p9"
Sep 11 00:58:16.094532 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:16.094174    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hl4x" podUID="526304d7-8bd3-4196-a695-f5845e8846a1"
Sep 11 00:58:16.094532 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:16.094262    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbpsj" podUID="ff3ee2a6-e101-4cde-8d89-489f1db81ab8"
Sep 11 00:58:16.094532 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:16.094362    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9s5p9" podUID="77093864-53b6-4044-952a-d8f1a965dd1d"
Sep 11 00:58:18.094135 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:18.094098    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hl4x"
Sep 11 00:58:18.094494 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:18.094194    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:58:18.094494 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:18.094202    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hl4x" podUID="526304d7-8bd3-4196-a695-f5845e8846a1"
Sep 11 00:58:18.094494 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:18.094224    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5p9"
Sep 11 00:58:18.094494 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:18.094293    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbpsj" podUID="ff3ee2a6-e101-4cde-8d89-489f1db81ab8"
Sep 11 00:58:18.094494 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:18.094354    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9s5p9" podUID="77093864-53b6-4044-952a-d8f1a965dd1d"
Sep 11 00:58:20.094561 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:20.094523    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:58:20.094561 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:20.094565    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hl4x"
Sep 11 00:58:20.095059 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:20.094660    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5p9"
Sep 11 00:58:20.095059 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:20.094665    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbpsj" podUID="ff3ee2a6-e101-4cde-8d89-489f1db81ab8"
Sep 11 00:58:20.095059 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:20.094732    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hl4x" podUID="526304d7-8bd3-4196-a695-f5845e8846a1"
Sep 11 00:58:20.095059 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:20.094830    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9s5p9" podUID="77093864-53b6-4044-952a-d8f1a965dd1d"
Sep 11 00:58:21.824613 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:21.824579    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77093864-53b6-4044-952a-d8f1a965dd1d-metrics-certs\") pod \"network-metrics-daemon-9s5p9\" (UID: \"77093864-53b6-4044-952a-d8f1a965dd1d\") " pod="openshift-multus/network-metrics-daemon-9s5p9"
Sep 11 00:58:21.825109 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:21.824675    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff3ee2a6-e101-4cde-8d89-489f1db81ab8-original-pull-secret\") pod \"global-pull-secret-syncer-jbpsj\" (UID: \"ff3ee2a6-e101-4cde-8d89-489f1db81ab8\") " pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:58:21.825109 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:21.824767    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 00:58:21.825109 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:21.824788    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 00:58:21.825109 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:21.824841    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77093864-53b6-4044-952a-d8f1a965dd1d-metrics-certs podName:77093864-53b6-4044-952a-d8f1a965dd1d nodeName:}" failed. No retries permitted until 2026-09-11 00:58:37.824819755 +0000 UTC m=+33.222011919 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/77093864-53b6-4044-952a-d8f1a965dd1d-metrics-certs") pod "network-metrics-daemon-9s5p9" (UID: "77093864-53b6-4044-952a-d8f1a965dd1d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 00:58:21.825109 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:21.824855    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff3ee2a6-e101-4cde-8d89-489f1db81ab8-original-pull-secret podName:ff3ee2a6-e101-4cde-8d89-489f1db81ab8 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:37.824849414 +0000 UTC m=+33.222041565 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ff3ee2a6-e101-4cde-8d89-489f1db81ab8-original-pull-secret") pod "global-pull-secret-syncer-jbpsj" (UID: "ff3ee2a6-e101-4cde-8d89-489f1db81ab8") : object "kube-system"/"original-pull-secret" not registered
Sep 11 00:58:21.925839 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:21.925808    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tdws\" (UniqueName: \"kubernetes.io/projected/526304d7-8bd3-4196-a695-f5845e8846a1-kube-api-access-2tdws\") pod \"network-check-target-9hl4x\" (UID: \"526304d7-8bd3-4196-a695-f5845e8846a1\") " pod="openshift-network-diagnostics/network-check-target-9hl4x"
Sep 11 00:58:21.925994 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:21.925965    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 00:58:21.925994 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:21.925983    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 00:58:21.925994 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:21.925992    2574 projected.go:196] Error preparing data for projected volume kube-api-access-2tdws for pod openshift-network-diagnostics/network-check-target-9hl4x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 00:58:21.926102 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:21.926040    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/526304d7-8bd3-4196-a695-f5845e8846a1-kube-api-access-2tdws podName:526304d7-8bd3-4196-a695-f5845e8846a1 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:37.92602573 +0000 UTC m=+33.323217883 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2tdws" (UniqueName: "kubernetes.io/projected/526304d7-8bd3-4196-a695-f5845e8846a1-kube-api-access-2tdws") pod "network-check-target-9hl4x" (UID: "526304d7-8bd3-4196-a695-f5845e8846a1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 00:58:22.094008 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:22.093939    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hl4x"
Sep 11 00:58:22.094008 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:22.093986    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:58:22.094188 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:22.094059    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5p9"
Sep 11 00:58:22.094188 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:22.094079    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hl4x" podUID="526304d7-8bd3-4196-a695-f5845e8846a1"
Sep 11 00:58:22.094188 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:22.094138    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9s5p9" podUID="77093864-53b6-4044-952a-d8f1a965dd1d"
Sep 11 00:58:22.094305 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:22.094224    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbpsj" podUID="ff3ee2a6-e101-4cde-8d89-489f1db81ab8"
Sep 11 00:58:24.094169 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:24.094128    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hl4x"
Sep 11 00:58:24.094169 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:24.094153    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:58:24.094169 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:24.094163    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5p9"
Sep 11 00:58:24.094734 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:24.094255    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hl4x" podUID="526304d7-8bd3-4196-a695-f5845e8846a1"
Sep 11 00:58:24.094734 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:24.094393    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbpsj" podUID="ff3ee2a6-e101-4cde-8d89-489f1db81ab8"
Sep 11 00:58:24.094734 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:24.094488    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9s5p9" podUID="77093864-53b6-4044-952a-d8f1a965dd1d"
Sep 11 00:58:26.094865 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:26.094796    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5p9"
Sep 11 00:58:26.095252 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:26.094894    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hl4x"
Sep 11 00:58:26.095252 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:26.094930    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:58:26.095252 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:26.094895    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9s5p9" podUID="77093864-53b6-4044-952a-d8f1a965dd1d"
Sep 11 00:58:26.095252 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:26.095014    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hl4x" podUID="526304d7-8bd3-4196-a695-f5845e8846a1"
Sep 11 00:58:26.095252 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:26.095086    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbpsj" podUID="ff3ee2a6-e101-4cde-8d89-489f1db81ab8"
Sep 11 00:58:27.365169 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:27.364795    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zw22q" event={"ID":"d097ee33-8900-43b7-9eae-35ac88373d1d","Type":"ContainerStarted","Data":"fc0f5345f5551549fd8e1209373a89b08fd8b3ecf8f04d633909c44547bde67c"}
Sep 11 00:58:27.367319 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:27.367294    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d7crj" event={"ID":"5176ce4f-e447-4f42-a338-2f1ee4761210","Type":"ContainerStarted","Data":"0dba8af39ae0e3371c9a6b0a19e92ac804d6fa9466030d4a05ba7e2346577279"}
Sep 11 00:58:27.367448 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:27.367369    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d7crj" event={"ID":"5176ce4f-e447-4f42-a338-2f1ee4761210","Type":"ContainerStarted","Data":"8b16dba3b94619917dc94ac7c36a143e8106d03e14a199ca8f2ad7036d442155"}
Sep 11 00:58:27.367448 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:27.367418    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d7crj" event={"ID":"5176ce4f-e447-4f42-a338-2f1ee4761210","Type":"ContainerStarted","Data":"6e77107885c85236c071bd79868284443a8ea8be4f6348843580eda3e3c2532f"}
Sep 11 00:58:27.367535 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:27.367482    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d7crj" event={"ID":"5176ce4f-e447-4f42-a338-2f1ee4761210","Type":"ContainerStarted","Data":"850407d8ec3704642f639bf37309f9b09049024eeddbf4446e312f9067ddeb7c"}
Sep 11 00:58:27.368727 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:27.368707    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-235.ec2.internal" event={"ID":"9f1f5adc45e045b22e89459332dfb6a0","Type":"ContainerStarted","Data":"5902ab133bf0877be03152098402c8bc464b543e3ef6db12289b4e210233e4ce"}
Sep 11 00:58:27.370079 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:27.370060    2574 generic.go:356] "Generic (PLEG): container finished" podID="d958aa85aad010b15ac136386e20db34" containerID="a2a4a2cceba6c29db2091867391a18207f49c58ad3de463018429857e844ff95" exitCode=0
Sep 11 00:58:27.370173 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:27.370131    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-235.ec2.internal" event={"ID":"d958aa85aad010b15ac136386e20db34","Type":"ContainerDied","Data":"a2a4a2cceba6c29db2091867391a18207f49c58ad3de463018429857e844ff95"}
Sep 11 00:58:27.371596 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:27.371578    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-w9q8q" event={"ID":"f4363dbb-d8eb-4eb6-af98-f3327b90c84c","Type":"ContainerStarted","Data":"562c93848fb942a59d4960c02880c04fd7b49e37323ae59e2e3201b00e39844a"}
Sep 11 00:58:27.373324 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:27.373295    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hh5wm" event={"ID":"1625cd1f-5711-4d05-9cfc-74a2a2c1c637","Type":"ContainerStarted","Data":"8e3e10be86fbbc95f3f579b331cf692b340e16e1d017f5aa7625b11fab84b5e7"}
Sep 11 00:58:27.374822 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:27.374798    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l5nhz" event={"ID":"396a7cf9-e334-407d-ad45-789fb8dfba05","Type":"ContainerStarted","Data":"fc8cc3fa64b7924d84165829e1a5af0be1d03dbd6debf82861471d3fa5d04026"}
Sep 11 00:58:27.378354 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:27.378330    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhmsk" event={"ID":"8ad98990-55b3-4499-9b32-a8b394175160","Type":"ContainerStarted","Data":"8f50dd29b777dd17ea36fee50bce8cae53c1edcfd587651dc5c393c0a1d17dd9"}
Sep 11 00:58:27.379124 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:27.379077    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-zw22q" podStartSLOduration=2.526886461 podStartE2EDuration="22.379066666s" totalImagesPullingTime="19.852180205s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 00:58:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:58:27.378441486 +0000 UTC m=+22.775633659" watchObservedRunningTime="2026-09-11 00:58:27.379066666 +0000 UTC m=+22.776258839"
Sep 11 00:58:27.380506 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:27.380489    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8q8bw" event={"ID":"390448c5-9242-4562-84e5-0307125ba1ad","Type":"ContainerStarted","Data":"c02c09be9283209189167def17bc8a5dfc1b828aed3abbcaa186445751598aa6"}
Sep 11 00:58:27.381973 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:27.381953    2574 generic.go:356] "Generic (PLEG): container finished" podID="1fe146d4-3a94-48d2-b383-ea0a4c301b54" containerID="fafe88e7fce6619be3c4f1027561133ff0bbdd21ca1e4c7961f82746e681921d" exitCode=0
Sep 11 00:58:27.382044 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:27.381981    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7j9mn" event={"ID":"1fe146d4-3a94-48d2-b383-ea0a4c301b54","Type":"ContainerDied","Data":"fafe88e7fce6619be3c4f1027561133ff0bbdd21ca1e4c7961f82746e681921d"}
Sep 11 00:58:27.391743 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:27.391712    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-l5nhz" podStartSLOduration=2.5982987619999998 podStartE2EDuration="22.39170098s" totalImagesPullingTime="19.793402218s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:58:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:58:27.39144513 +0000 UTC m=+22.788637301" watchObservedRunningTime="2026-09-11 00:58:27.39170098 +0000 UTC m=+22.788893144"
Sep 11 00:58:27.406063 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:27.405988    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-hh5wm" podStartSLOduration=2.571800974 podStartE2EDuration="22.40597209s" totalImagesPullingTime="19.834171116s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 00:58:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:58:27.405629901 +0000 UTC m=+22.802822075" watchObservedRunningTime="2026-09-11 00:58:27.40597209 +0000 UTC m=+22.803164267"
Sep 11 00:58:27.422354 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:27.422313    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-w9q8q" podStartSLOduration=2.461062124 podStartE2EDuration="22.422300126s" totalImagesPullingTime="19.961238002s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 00:58:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:58:27.421848849 +0000 UTC m=+22.819041023" watchObservedRunningTime="2026-09-11 00:58:27.422300126 +0000 UTC m=+22.819492298"
Sep 11 00:58:27.467385 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:27.467339    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-235.ec2.internal" podStartSLOduration=21.46732596 podStartE2EDuration="21.46732596s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 00:58:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:58:27.448243743 +0000 UTC m=+22.845435917" watchObservedRunningTime="2026-09-11 00:58:27.46732596 +0000 UTC m=+22.864518186"
Sep 11 00:58:27.485325 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:27.485277    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8q8bw" podStartSLOduration=2.478651528 podStartE2EDuration="22.485258497s" totalImagesPullingTime="20.006606969s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 00:58:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:58:27.484474817 +0000 UTC m=+22.881666990" watchObservedRunningTime="2026-09-11 00:58:27.485258497 +0000 UTC m=+22.882450671"
Sep 11 00:58:27.490670 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:27.490634    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:27.559180 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:27.559155    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:27.658931 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:27.658427    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:28.068860 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:28.068838    2574 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 11 00:58:28.093988 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:28.093964    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5p9"
Sep 11 00:58:28.094109 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:28.093986    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hl4x"
Sep 11 00:58:28.094109 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:28.093986    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:58:28.094109 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:28.094094    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9s5p9" podUID="77093864-53b6-4044-952a-d8f1a965dd1d"
Sep 11 00:58:28.094265 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:28.094223    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hl4x" podUID="526304d7-8bd3-4196-a695-f5845e8846a1"
Sep 11 00:58:28.094333 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:28.094316    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbpsj" podUID="ff3ee2a6-e101-4cde-8d89-489f1db81ab8"
Sep 11 00:58:28.387603 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:28.387525    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qx27q" event={"ID":"c29dcdd7-781d-411d-b6e1-4cb59de9be56","Type":"ContainerStarted","Data":"2fed12032db6bb95e55268970b9f8d2d093bd33494536aa762b266d0c9ec48aa"}
Sep 11 00:58:28.389780 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:28.389756    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhmsk" event={"ID":"8ad98990-55b3-4499-9b32-a8b394175160","Type":"ContainerStarted","Data":"0ee8a35932549b695f74ea0fa8adcc59984795c32599692610d4d4d3553ced76"}
Sep 11 00:58:28.393443 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:28.393414    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d7crj" event={"ID":"5176ce4f-e447-4f42-a338-2f1ee4761210","Type":"ContainerStarted","Data":"01f242892dffe7b590b66534cf722a5e15a68c27d4fc0927ccc4250cda463e91"}
Sep 11 00:58:28.393572 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:28.393473    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d7crj" event={"ID":"5176ce4f-e447-4f42-a338-2f1ee4761210","Type":"ContainerStarted","Data":"30b928c5c0fe0b3b46c7add056229d4a387b3980ca221c80a53d59620599ecd0"}
Sep 11 00:58:28.395354 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:28.395329    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-235.ec2.internal" event={"ID":"d958aa85aad010b15ac136386e20db34","Type":"ContainerStarted","Data":"e30b0e817684198a443e66a5ab86ad45469c4ba61fc4d67d3daa14fa570e7503"}
Sep 11 00:58:28.436672 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:28.436625    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qx27q" podStartSLOduration=3.463737478 podStartE2EDuration="23.436609834s" totalImagesPullingTime="19.972872356s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 00:58:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:58:28.417975692 +0000 UTC m=+23.815167865" watchObservedRunningTime="2026-09-11 00:58:28.436609834 +0000 UTC m=+23.833802004"
Sep 11 00:58:28.552692 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:28.552641    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:28.654646 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:28.654563    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:28.752208 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:28.752133    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:28.853071 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:28.853030    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:28.959157 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:28.959057    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:29.040584 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:29.040549    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:29.054551 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:29.054456    2574 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T00:58:28.06885754Z","UUID":"4c209aaa-2a9d-478d-8400-6b3b538e3a56","Handler":null,"Name":"","Endpoint":""}
Sep 11 00:58:29.056095 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:29.056068    2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 11 00:58:29.056095 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:29.056092    2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 11 00:58:29.400068 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:29.400012    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhmsk" event={"ID":"8ad98990-55b3-4499-9b32-a8b394175160","Type":"ContainerStarted","Data":"01134639082b7e85e6babb503dbf56c3a08a71a1e8ce3a450362ecceb4bf1462"}
Sep 11 00:58:29.418117 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:29.418069    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-235.ec2.internal" podStartSLOduration=23.418053219 podStartE2EDuration="23.418053219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 00:58:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:58:28.437150591 +0000 UTC m=+23.834342764" watchObservedRunningTime="2026-09-11 00:58:29.418053219 +0000 UTC m=+24.815245392"
Sep 11 00:58:29.418361 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:29.418337    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhmsk" podStartSLOduration=2.961109272 podStartE2EDuration="24.418328421s" totalImagesPullingTime="21.457219149s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 00:58:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:58:29.417758189 +0000 UTC m=+24.814950361" watchObservedRunningTime="2026-09-11 00:58:29.418328421 +0000 UTC m=+24.815520590"
Sep 11 00:58:30.094456 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:30.094357    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:58:30.094597 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:30.094361    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5p9"
Sep 11 00:58:30.094597 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:30.094484    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbpsj" podUID="ff3ee2a6-e101-4cde-8d89-489f1db81ab8"
Sep 11 00:58:30.094597 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:30.094492    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hl4x"
Sep 11 00:58:30.094756 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:30.094599    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9s5p9" podUID="77093864-53b6-4044-952a-d8f1a965dd1d"
Sep 11 00:58:30.094756 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:30.094682    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hl4x" podUID="526304d7-8bd3-4196-a695-f5845e8846a1"
Sep 11 00:58:30.406807 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:30.406734    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d7crj" event={"ID":"5176ce4f-e447-4f42-a338-2f1ee4761210","Type":"ContainerStarted","Data":"500ee6ea31ecc9d69c92007b8ab5da5cc6bf2cbd93843eadf44dbb5e865dc19c"}
Sep 11 00:58:31.162587 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:31.162551    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-l5nhz"
Sep 11 00:58:31.163195 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:31.163178    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-l5nhz"
Sep 11 00:58:31.408928 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:31.408889    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-l5nhz"
Sep 11 00:58:31.409484 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:31.409466    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-l5nhz"
Sep 11 00:58:31.487350 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:31.487281    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:32.094464 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:32.094207    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5p9"
Sep 11 00:58:32.094603 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:32.094283    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:58:32.094603 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:32.094529    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9s5p9" podUID="77093864-53b6-4044-952a-d8f1a965dd1d"
Sep 11 00:58:32.094603 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:32.094290    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hl4x"
Sep 11 00:58:32.094603 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:32.094570    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbpsj" podUID="ff3ee2a6-e101-4cde-8d89-489f1db81ab8"
Sep 11 00:58:32.094797 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:32.094634    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hl4x" podUID="526304d7-8bd3-4196-a695-f5845e8846a1"
Sep 11 00:58:32.412789 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:32.412761    2574 generic.go:356] "Generic (PLEG): container finished" podID="1fe146d4-3a94-48d2-b383-ea0a4c301b54" containerID="73f95677355a9fe1312f72704a7a510d81ce9d227a09661680638f1726999bfb" exitCode=0
Sep 11 00:58:32.413231 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:32.412854    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7j9mn" event={"ID":"1fe146d4-3a94-48d2-b383-ea0a4c301b54","Type":"ContainerDied","Data":"73f95677355a9fe1312f72704a7a510d81ce9d227a09661680638f1726999bfb"}
Sep 11 00:58:32.416868 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:32.416846    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d7crj" event={"ID":"5176ce4f-e447-4f42-a338-2f1ee4761210","Type":"ContainerStarted","Data":"28ed2db4f9460d5ad491a7f5a00282fcbb275c116b3b1e6e061b77e51001dd35"}
Sep 11 00:58:32.417165 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:32.417120    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:32.417298 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:32.417186    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:32.417298 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:32.417239    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:32.433728 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:32.433502    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:32.433865 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:32.433744    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:58:32.457936 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:32.457882    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-d7crj" podStartSLOduration=6.99494019 podStartE2EDuration="27.457871152s" totalImagesPullingTime="20.462930962s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 00:58:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:58:32.456834767 +0000 UTC m=+27.854026939" watchObservedRunningTime="2026-09-11 00:58:32.457871152 +0000 UTC m=+27.855063325"
Sep 11 00:58:33.395587 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:33.395559    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:33.451160 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:33.451137    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:33.890453 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:33.890269    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9hl4x"]
Sep 11 00:58:33.890619 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:33.890577    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hl4x"
Sep 11 00:58:33.890692 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:33.890667    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hl4x" podUID="526304d7-8bd3-4196-a695-f5845e8846a1"
Sep 11 00:58:33.890937 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:33.890896    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jbpsj"]
Sep 11 00:58:33.891032 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:33.891019    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:58:33.891133 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:33.891115    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbpsj" podUID="ff3ee2a6-e101-4cde-8d89-489f1db81ab8"
Sep 11 00:58:33.901591 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:33.901535    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9s5p9"]
Sep 11 00:58:33.901693 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:33.901639    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5p9"
Sep 11 00:58:33.901743 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:33.901728    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9s5p9" podUID="77093864-53b6-4044-952a-d8f1a965dd1d"
Sep 11 00:58:34.423343 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:34.423308    2574 generic.go:356] "Generic (PLEG): container finished" podID="1fe146d4-3a94-48d2-b383-ea0a4c301b54" containerID="6c20a3ae3128987ac03b65b9612c31ff4c81af710831202f989aae107309ed82" exitCode=0
Sep 11 00:58:34.423486 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:34.423400    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7j9mn" event={"ID":"1fe146d4-3a94-48d2-b383-ea0a4c301b54","Type":"ContainerDied","Data":"6c20a3ae3128987ac03b65b9612c31ff4c81af710831202f989aae107309ed82"}
Sep 11 00:58:35.482599 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:35.482578    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:36.094926 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:36.094897    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hl4x"
Sep 11 00:58:36.095090 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:36.094896    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5p9"
Sep 11 00:58:36.095090 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:36.095023    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hl4x" podUID="526304d7-8bd3-4196-a695-f5845e8846a1"
Sep 11 00:58:36.095090 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:36.094897    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:58:36.095090 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:36.095079    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9s5p9" podUID="77093864-53b6-4044-952a-d8f1a965dd1d"
Sep 11 00:58:36.095293 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:36.095133    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbpsj" podUID="ff3ee2a6-e101-4cde-8d89-489f1db81ab8"
Sep 11 00:58:36.429443 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:36.429404    2574 generic.go:356] "Generic (PLEG): container finished" podID="1fe146d4-3a94-48d2-b383-ea0a4c301b54" containerID="db613d28cdda1f6b73638416cce8debe64fb2a51f45f5ef40bb40396e2dca946" exitCode=0
Sep 11 00:58:36.429583 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:36.429471    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7j9mn" event={"ID":"1fe146d4-3a94-48d2-b383-ea0a4c301b54","Type":"ContainerDied","Data":"db613d28cdda1f6b73638416cce8debe64fb2a51f45f5ef40bb40396e2dca946"}
Sep 11 00:58:37.848122 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:37.848086    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff3ee2a6-e101-4cde-8d89-489f1db81ab8-original-pull-secret\") pod \"global-pull-secret-syncer-jbpsj\" (UID: \"ff3ee2a6-e101-4cde-8d89-489f1db81ab8\") " pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:58:37.848574 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:37.848132    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77093864-53b6-4044-952a-d8f1a965dd1d-metrics-certs\") pod \"network-metrics-daemon-9s5p9\" (UID: \"77093864-53b6-4044-952a-d8f1a965dd1d\") " pod="openshift-multus/network-metrics-daemon-9s5p9"
Sep 11 00:58:37.848574 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:37.848242    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 00:58:37.848574 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:37.848262    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 00:58:37.848574 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:37.848316    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77093864-53b6-4044-952a-d8f1a965dd1d-metrics-certs podName:77093864-53b6-4044-952a-d8f1a965dd1d nodeName:}" failed. No retries permitted until 2026-09-11 00:59:09.848298 +0000 UTC m=+65.245490164 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/77093864-53b6-4044-952a-d8f1a965dd1d-metrics-certs") pod "network-metrics-daemon-9s5p9" (UID: "77093864-53b6-4044-952a-d8f1a965dd1d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 00:58:37.848574 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:37.848333    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff3ee2a6-e101-4cde-8d89-489f1db81ab8-original-pull-secret podName:ff3ee2a6-e101-4cde-8d89-489f1db81ab8 nodeName:}" failed. No retries permitted until 2026-09-11 00:59:09.848325178 +0000 UTC m=+65.245517334 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ff3ee2a6-e101-4cde-8d89-489f1db81ab8-original-pull-secret") pod "global-pull-secret-syncer-jbpsj" (UID: "ff3ee2a6-e101-4cde-8d89-489f1db81ab8") : object "kube-system"/"original-pull-secret" not registered
Sep 11 00:58:37.949446 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:37.949418    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tdws\" (UniqueName: \"kubernetes.io/projected/526304d7-8bd3-4196-a695-f5845e8846a1-kube-api-access-2tdws\") pod \"network-check-target-9hl4x\" (UID: \"526304d7-8bd3-4196-a695-f5845e8846a1\") " pod="openshift-network-diagnostics/network-check-target-9hl4x"
Sep 11 00:58:37.949602 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:37.949550    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 00:58:37.949602 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:37.949563    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 00:58:37.949602 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:37.949575    2574 projected.go:196] Error preparing data for projected volume kube-api-access-2tdws for pod openshift-network-diagnostics/network-check-target-9hl4x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 00:58:37.949759 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:37.949625    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/526304d7-8bd3-4196-a695-f5845e8846a1-kube-api-access-2tdws podName:526304d7-8bd3-4196-a695-f5845e8846a1 nodeName:}" failed. No retries permitted until 2026-09-11 00:59:09.949607574 +0000 UTC m=+65.346799745 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-2tdws" (UniqueName: "kubernetes.io/projected/526304d7-8bd3-4196-a695-f5845e8846a1-kube-api-access-2tdws") pod "network-check-target-9hl4x" (UID: "526304d7-8bd3-4196-a695-f5845e8846a1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 00:58:38.094682 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:38.094646    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:58:38.094845 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:38.094646    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hl4x"
Sep 11 00:58:38.094845 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:38.094769    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbpsj" podUID="ff3ee2a6-e101-4cde-8d89-489f1db81ab8"
Sep 11 00:58:38.095021 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:38.094856    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hl4x" podUID="526304d7-8bd3-4196-a695-f5845e8846a1"
Sep 11 00:58:38.095021 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:38.094651    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5p9"
Sep 11 00:58:38.095021 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:38.094961    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9s5p9" podUID="77093864-53b6-4044-952a-d8f1a965dd1d"
Sep 11 00:58:39.434336 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.434304    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-235.ec2.internal" event="NodeReady"
Sep 11 00:58:39.434795 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.434470    2574 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 11 00:58:39.491791 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.491715    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-cs2xq"]
Sep 11 00:58:39.494131 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.494102    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cs2xq"
Sep 11 00:58:39.497023 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.496531    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 11 00:58:39.497023 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.496577    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 11 00:58:39.497023 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.496635    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vpvqt\""
Sep 11 00:58:39.497023 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.496853    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 11 00:58:39.499327 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.499295    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-j9mpz"]
Sep 11 00:58:39.500982 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.500962    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-j9mpz"
Sep 11 00:58:39.503621 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.503603    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xx7m2\""
Sep 11 00:58:39.503895 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.503879    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 11 00:58:39.504101 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.504086    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 11 00:58:39.509437 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.509419    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cs2xq"]
Sep 11 00:58:39.510125 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.510075    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:39.513744 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.513725    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-j9mpz"]
Sep 11 00:58:39.577612 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.577577    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:39.642172 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.642136    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:39.661468 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.661444    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sflmn\" (UniqueName: \"kubernetes.io/projected/af240b86-c806-4cf9-a3f0-0bee21e7abfe-kube-api-access-sflmn\") pod \"ingress-canary-cs2xq\" (UID: \"af240b86-c806-4cf9-a3f0-0bee21e7abfe\") " pod="openshift-ingress-canary/ingress-canary-cs2xq"
Sep 11 00:58:39.661610 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.661491    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dda2c7fa-a50c-483c-8709-775d04d924c2-config-volume\") pod \"dns-default-j9mpz\" (UID: \"dda2c7fa-a50c-483c-8709-775d04d924c2\") " pod="openshift-dns/dns-default-j9mpz"
Sep 11 00:58:39.661610 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.661572    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/af240b86-c806-4cf9-a3f0-0bee21e7abfe-cert\") pod \"ingress-canary-cs2xq\" (UID: \"af240b86-c806-4cf9-a3f0-0bee21e7abfe\") " pod="openshift-ingress-canary/ingress-canary-cs2xq"
Sep 11 00:58:39.661610 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.661605    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dda2c7fa-a50c-483c-8709-775d04d924c2-tmp-dir\") pod \"dns-default-j9mpz\" (UID: \"dda2c7fa-a50c-483c-8709-775d04d924c2\") " pod="openshift-dns/dns-default-j9mpz"
Sep 11 00:58:39.661754 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.661639    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dda2c7fa-a50c-483c-8709-775d04d924c2-metrics-tls\") pod \"dns-default-j9mpz\" (UID: \"dda2c7fa-a50c-483c-8709-775d04d924c2\") " pod="openshift-dns/dns-default-j9mpz"
Sep 11 00:58:39.661754 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.661712    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmzp9\" (UniqueName: \"kubernetes.io/projected/dda2c7fa-a50c-483c-8709-775d04d924c2-kube-api-access-cmzp9\") pod \"dns-default-j9mpz\" (UID: \"dda2c7fa-a50c-483c-8709-775d04d924c2\") " pod="openshift-dns/dns-default-j9mpz"
Sep 11 00:58:39.695624 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.695599    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:39.762282 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.762216    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmzp9\" (UniqueName: \"kubernetes.io/projected/dda2c7fa-a50c-483c-8709-775d04d924c2-kube-api-access-cmzp9\") pod \"dns-default-j9mpz\" (UID: \"dda2c7fa-a50c-483c-8709-775d04d924c2\") " pod="openshift-dns/dns-default-j9mpz"
Sep 11 00:58:39.762282 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.762249    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sflmn\" (UniqueName: \"kubernetes.io/projected/af240b86-c806-4cf9-a3f0-0bee21e7abfe-kube-api-access-sflmn\") pod \"ingress-canary-cs2xq\" (UID: \"af240b86-c806-4cf9-a3f0-0bee21e7abfe\") " pod="openshift-ingress-canary/ingress-canary-cs2xq"
Sep 11 00:58:39.762473 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.762297    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dda2c7fa-a50c-483c-8709-775d04d924c2-config-volume\") pod \"dns-default-j9mpz\" (UID: \"dda2c7fa-a50c-483c-8709-775d04d924c2\") " pod="openshift-dns/dns-default-j9mpz"
Sep 11 00:58:39.762473 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.762341    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/af240b86-c806-4cf9-a3f0-0bee21e7abfe-cert\") pod \"ingress-canary-cs2xq\" (UID: \"af240b86-c806-4cf9-a3f0-0bee21e7abfe\") " pod="openshift-ingress-canary/ingress-canary-cs2xq"
Sep 11 00:58:39.762569 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.762514    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dda2c7fa-a50c-483c-8709-775d04d924c2-tmp-dir\") pod \"dns-default-j9mpz\" (UID: \"dda2c7fa-a50c-483c-8709-775d04d924c2\") " pod="openshift-dns/dns-default-j9mpz"
Sep 11 00:58:39.762619 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.762564    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dda2c7fa-a50c-483c-8709-775d04d924c2-metrics-tls\") pod \"dns-default-j9mpz\" (UID: \"dda2c7fa-a50c-483c-8709-775d04d924c2\") " pod="openshift-dns/dns-default-j9mpz"
Sep 11 00:58:39.762702 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:39.762673    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 00:58:39.762778 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:39.762738    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dda2c7fa-a50c-483c-8709-775d04d924c2-metrics-tls podName:dda2c7fa-a50c-483c-8709-775d04d924c2 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:40.26271968 +0000 UTC m=+35.659911845 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dda2c7fa-a50c-483c-8709-775d04d924c2-metrics-tls") pod "dns-default-j9mpz" (UID: "dda2c7fa-a50c-483c-8709-775d04d924c2") : secret "dns-default-metrics-tls" not found
Sep 11 00:58:39.762874 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.762853    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dda2c7fa-a50c-483c-8709-775d04d924c2-tmp-dir\") pod \"dns-default-j9mpz\" (UID: \"dda2c7fa-a50c-483c-8709-775d04d924c2\") " pod="openshift-dns/dns-default-j9mpz"
Sep 11 00:58:39.763085 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.763066    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dda2c7fa-a50c-483c-8709-775d04d924c2-config-volume\") pod \"dns-default-j9mpz\" (UID: \"dda2c7fa-a50c-483c-8709-775d04d924c2\") " pod="openshift-dns/dns-default-j9mpz"
Sep 11 00:58:39.767389 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.767368    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/af240b86-c806-4cf9-a3f0-0bee21e7abfe-cert\") pod \"ingress-canary-cs2xq\" (UID: \"af240b86-c806-4cf9-a3f0-0bee21e7abfe\") " pod="openshift-ingress-canary/ingress-canary-cs2xq"
Sep 11 00:58:39.771527 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.771499    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmzp9\" (UniqueName: \"kubernetes.io/projected/dda2c7fa-a50c-483c-8709-775d04d924c2-kube-api-access-cmzp9\") pod \"dns-default-j9mpz\" (UID: \"dda2c7fa-a50c-483c-8709-775d04d924c2\") " pod="openshift-dns/dns-default-j9mpz"
Sep 11 00:58:39.771622 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.771565    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sflmn\" (UniqueName: \"kubernetes.io/projected/af240b86-c806-4cf9-a3f0-0bee21e7abfe-kube-api-access-sflmn\") pod \"ingress-canary-cs2xq\" (UID: \"af240b86-c806-4cf9-a3f0-0bee21e7abfe\") " pod="openshift-ingress-canary/ingress-canary-cs2xq"
Sep 11 00:58:39.805548 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.805521    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cs2xq"
Sep 11 00:58:39.860581 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.860550    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:39.924982 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:39.924953    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:40.094655 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:40.094576    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:58:40.094816 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:40.094583    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5p9"
Sep 11 00:58:40.095200 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:40.094590    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hl4x"
Sep 11 00:58:40.097199 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:40.097178    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 00:58:40.098005 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:40.097983    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 00:58:40.098005 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:40.097998    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zpsrm\""
Sep 11 00:58:40.098169 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:40.098012    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-h86gq\""
Sep 11 00:58:40.098169 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:40.098036    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 00:58:40.098169 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:40.097997    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 00:58:40.268127 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:40.268090    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dda2c7fa-a50c-483c-8709-775d04d924c2-metrics-tls\") pod \"dns-default-j9mpz\" (UID: \"dda2c7fa-a50c-483c-8709-775d04d924c2\") " pod="openshift-dns/dns-default-j9mpz"
Sep 11 00:58:40.268294 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:40.268226    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 00:58:40.268294 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:40.268291    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dda2c7fa-a50c-483c-8709-775d04d924c2-metrics-tls podName:dda2c7fa-a50c-483c-8709-775d04d924c2 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:41.268273094 +0000 UTC m=+36.665465245 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dda2c7fa-a50c-483c-8709-775d04d924c2-metrics-tls") pod "dns-default-j9mpz" (UID: "dda2c7fa-a50c-483c-8709-775d04d924c2") : secret "dns-default-metrics-tls" not found
Sep 11 00:58:40.441683 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:40.441525    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7j9mn" event={"ID":"1fe146d4-3a94-48d2-b383-ea0a4c301b54","Type":"ContainerStarted","Data":"05ecd190c3b7c00a76154748ed788cf83c6e5370c531cb5d6cacf9677212d271"}
Sep 11 00:58:40.558999 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:40.558977    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cs2xq"]
Sep 11 00:58:40.562621 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:40.562593    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf240b86_c806_4cf9_a3f0_0bee21e7abfe.slice/crio-2cc87c203b07c5ccf861b1314f680835be0b9320d954d14d3afa65f9430b06f2 WatchSource:0}: Error finding container 2cc87c203b07c5ccf861b1314f680835be0b9320d954d14d3afa65f9430b06f2: Status 404 returned error can't find the container with id 2cc87c203b07c5ccf861b1314f680835be0b9320d954d14d3afa65f9430b06f2
Sep 11 00:58:41.274757 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:41.274720    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dda2c7fa-a50c-483c-8709-775d04d924c2-metrics-tls\") pod \"dns-default-j9mpz\" (UID: \"dda2c7fa-a50c-483c-8709-775d04d924c2\") " pod="openshift-dns/dns-default-j9mpz"
Sep 11 00:58:41.274964 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:41.274948    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 00:58:41.275019 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:41.275013    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dda2c7fa-a50c-483c-8709-775d04d924c2-metrics-tls podName:dda2c7fa-a50c-483c-8709-775d04d924c2 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:43.274993182 +0000 UTC m=+38.672185341 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dda2c7fa-a50c-483c-8709-775d04d924c2-metrics-tls") pod "dns-default-j9mpz" (UID: "dda2c7fa-a50c-483c-8709-775d04d924c2") : secret "dns-default-metrics-tls" not found
Sep 11 00:58:41.447519 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:41.447485    2574 generic.go:356] "Generic (PLEG): container finished" podID="1fe146d4-3a94-48d2-b383-ea0a4c301b54" containerID="05ecd190c3b7c00a76154748ed788cf83c6e5370c531cb5d6cacf9677212d271" exitCode=0
Sep 11 00:58:41.448084 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:41.447561    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7j9mn" event={"ID":"1fe146d4-3a94-48d2-b383-ea0a4c301b54","Type":"ContainerDied","Data":"05ecd190c3b7c00a76154748ed788cf83c6e5370c531cb5d6cacf9677212d271"}
Sep 11 00:58:41.449286 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:41.449250    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cs2xq" event={"ID":"af240b86-c806-4cf9-a3f0-0bee21e7abfe","Type":"ContainerStarted","Data":"2cc87c203b07c5ccf861b1314f680835be0b9320d954d14d3afa65f9430b06f2"}
Sep 11 00:58:41.780258 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:41.780165    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:41.901612 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:41.901574    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:42.455578 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:42.455539    2574 generic.go:356] "Generic (PLEG): container finished" podID="1fe146d4-3a94-48d2-b383-ea0a4c301b54" containerID="5bc8407e7b43822488664cc40944227c3f8dab3ce83231d208fd5d14e38cc5a1" exitCode=0
Sep 11 00:58:42.456116 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:42.455607    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7j9mn" event={"ID":"1fe146d4-3a94-48d2-b383-ea0a4c301b54","Type":"ContainerDied","Data":"5bc8407e7b43822488664cc40944227c3f8dab3ce83231d208fd5d14e38cc5a1"}
Sep 11 00:58:42.501761 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:42.501728    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:42.579982 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:42.579961    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:42.764229 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:42.764207    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-v9q8v"]
Sep 11 00:58:42.779809 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:42.779781    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-v9q8v"]
Sep 11 00:58:42.779953 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:42.779878    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-v9q8v"
Sep 11 00:58:42.782216 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:42.782158    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 11 00:58:42.782314 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:42.782163    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 11 00:58:42.782314 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:42.782243    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 11 00:58:42.782314 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:42.782260    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mtm2j\""
Sep 11 00:58:42.782607 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:42.782591    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 11 00:58:42.782959 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:42.782939    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:42.843919 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:42.843861    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:42.885102 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:42.885080    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/000696c7-5f3f-469b-a8bd-3df2d07590b4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v9q8v\" (UID: \"000696c7-5f3f-469b-a8bd-3df2d07590b4\") " pod="openshift-insights/insights-runtime-extractor-v9q8v"
Sep 11 00:58:42.885189 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:42.885109    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwg2l\" (UniqueName: \"kubernetes.io/projected/000696c7-5f3f-469b-a8bd-3df2d07590b4-kube-api-access-gwg2l\") pod \"insights-runtime-extractor-v9q8v\" (UID: \"000696c7-5f3f-469b-a8bd-3df2d07590b4\") " pod="openshift-insights/insights-runtime-extractor-v9q8v"
Sep 11 00:58:42.885189 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:42.885131    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/000696c7-5f3f-469b-a8bd-3df2d07590b4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v9q8v\" (UID: \"000696c7-5f3f-469b-a8bd-3df2d07590b4\") " pod="openshift-insights/insights-runtime-extractor-v9q8v"
Sep 11 00:58:42.885269 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:42.885182    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/000696c7-5f3f-469b-a8bd-3df2d07590b4-crio-socket\") pod \"insights-runtime-extractor-v9q8v\" (UID: \"000696c7-5f3f-469b-a8bd-3df2d07590b4\") " pod="openshift-insights/insights-runtime-extractor-v9q8v"
Sep 11 00:58:42.885269 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:42.885218    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/000696c7-5f3f-469b-a8bd-3df2d07590b4-data-volume\") pod \"insights-runtime-extractor-v9q8v\" (UID: \"000696c7-5f3f-469b-a8bd-3df2d07590b4\") " pod="openshift-insights/insights-runtime-extractor-v9q8v"
Sep 11 00:58:42.986399 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:42.986339    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/000696c7-5f3f-469b-a8bd-3df2d07590b4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v9q8v\" (UID: \"000696c7-5f3f-469b-a8bd-3df2d07590b4\") " pod="openshift-insights/insights-runtime-extractor-v9q8v"
Sep 11 00:58:42.986399 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:42.986394    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwg2l\" (UniqueName: \"kubernetes.io/projected/000696c7-5f3f-469b-a8bd-3df2d07590b4-kube-api-access-gwg2l\") pod \"insights-runtime-extractor-v9q8v\" (UID: \"000696c7-5f3f-469b-a8bd-3df2d07590b4\") " pod="openshift-insights/insights-runtime-extractor-v9q8v"
Sep 11 00:58:42.986623 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:42.986419    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/000696c7-5f3f-469b-a8bd-3df2d07590b4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v9q8v\" (UID: \"000696c7-5f3f-469b-a8bd-3df2d07590b4\") " pod="openshift-insights/insights-runtime-extractor-v9q8v"
Sep 11 00:58:42.986623 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:42.986444    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/000696c7-5f3f-469b-a8bd-3df2d07590b4-crio-socket\") pod \"insights-runtime-extractor-v9q8v\" (UID: \"000696c7-5f3f-469b-a8bd-3df2d07590b4\") " pod="openshift-insights/insights-runtime-extractor-v9q8v"
Sep 11 00:58:42.986623 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:42.986462    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/000696c7-5f3f-469b-a8bd-3df2d07590b4-data-volume\") pod \"insights-runtime-extractor-v9q8v\" (UID: \"000696c7-5f3f-469b-a8bd-3df2d07590b4\") " pod="openshift-insights/insights-runtime-extractor-v9q8v"
Sep 11 00:58:42.986623 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:42.986511    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 00:58:42.986623 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:42.986582    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/000696c7-5f3f-469b-a8bd-3df2d07590b4-insights-runtime-extractor-tls podName:000696c7-5f3f-469b-a8bd-3df2d07590b4 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:43.486562187 +0000 UTC m=+38.883754352 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/000696c7-5f3f-469b-a8bd-3df2d07590b4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v9q8v" (UID: "000696c7-5f3f-469b-a8bd-3df2d07590b4") : secret "insights-runtime-extractor-tls" not found
Sep 11 00:58:42.986623 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:42.986591    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 00:58:42.986864 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:42.986666    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/000696c7-5f3f-469b-a8bd-3df2d07590b4-kube-rbac-proxy-cm podName:000696c7-5f3f-469b-a8bd-3df2d07590b4 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:43.486646768 +0000 UTC m=+38.883838932 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/000696c7-5f3f-469b-a8bd-3df2d07590b4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v9q8v" (UID: "000696c7-5f3f-469b-a8bd-3df2d07590b4") : configmap "kube-rbac-proxy" not found
Sep 11 00:58:42.986864 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:42.986730    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/000696c7-5f3f-469b-a8bd-3df2d07590b4-crio-socket\") pod \"insights-runtime-extractor-v9q8v\" (UID: \"000696c7-5f3f-469b-a8bd-3df2d07590b4\") " pod="openshift-insights/insights-runtime-extractor-v9q8v"
Sep 11 00:58:42.986864 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:42.986751    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/000696c7-5f3f-469b-a8bd-3df2d07590b4-data-volume\") pod \"insights-runtime-extractor-v9q8v\" (UID: \"000696c7-5f3f-469b-a8bd-3df2d07590b4\") " pod="openshift-insights/insights-runtime-extractor-v9q8v"
Sep 11 00:58:42.995271 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:42.995247    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwg2l\" (UniqueName: \"kubernetes.io/projected/000696c7-5f3f-469b-a8bd-3df2d07590b4-kube-api-access-gwg2l\") pod \"insights-runtime-extractor-v9q8v\" (UID: \"000696c7-5f3f-469b-a8bd-3df2d07590b4\") " pod="openshift-insights/insights-runtime-extractor-v9q8v"
Sep 11 00:58:43.233020 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:43.232990    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:43.289205 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:43.289033    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dda2c7fa-a50c-483c-8709-775d04d924c2-metrics-tls\") pod \"dns-default-j9mpz\" (UID: \"dda2c7fa-a50c-483c-8709-775d04d924c2\") " pod="openshift-dns/dns-default-j9mpz"
Sep 11 00:58:43.289333 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:43.289181    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 00:58:43.289333 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:43.289330    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dda2c7fa-a50c-483c-8709-775d04d924c2-metrics-tls podName:dda2c7fa-a50c-483c-8709-775d04d924c2 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:47.289313155 +0000 UTC m=+42.686505305 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dda2c7fa-a50c-483c-8709-775d04d924c2-metrics-tls") pod "dns-default-j9mpz" (UID: "dda2c7fa-a50c-483c-8709-775d04d924c2") : secret "dns-default-metrics-tls" not found
Sep 11 00:58:43.331879 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:43.331847    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:43.461975 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:43.461940    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7j9mn" event={"ID":"1fe146d4-3a94-48d2-b383-ea0a4c301b54","Type":"ContainerStarted","Data":"0f8353da4ae0ac56826da02242115951e8d174eda233fede393f734fbf263187"}
Sep 11 00:58:43.463542 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:43.463518    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cs2xq" event={"ID":"af240b86-c806-4cf9-a3f0-0bee21e7abfe","Type":"ContainerStarted","Data":"647f8951064a90d8a7fc048980bc81c5e93c206329da9f3f7e0cc33d0ffb2cd9"}
Sep 11 00:58:43.481568 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:43.481527    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7j9mn" podStartSLOduration=8.446487096 podStartE2EDuration="38.481514656s" totalImagesPullingTime="30.03502756s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 00:58:05 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:58:43.481055897 +0000 UTC m=+38.878248073" watchObservedRunningTime="2026-09-11 00:58:43.481514656 +0000 UTC m=+38.878706829"
Sep 11 00:58:43.490490 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:43.490439    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/000696c7-5f3f-469b-a8bd-3df2d07590b4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v9q8v\" (UID: \"000696c7-5f3f-469b-a8bd-3df2d07590b4\") " pod="openshift-insights/insights-runtime-extractor-v9q8v"
Sep 11 00:58:43.490490 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:43.490471    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/000696c7-5f3f-469b-a8bd-3df2d07590b4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v9q8v\" (UID: \"000696c7-5f3f-469b-a8bd-3df2d07590b4\") " pod="openshift-insights/insights-runtime-extractor-v9q8v"
Sep 11 00:58:43.490640 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:43.490536    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 00:58:43.490640 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:43.490591    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 00:58:43.490724 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:43.490598    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/000696c7-5f3f-469b-a8bd-3df2d07590b4-kube-rbac-proxy-cm podName:000696c7-5f3f-469b-a8bd-3df2d07590b4 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:44.490585292 +0000 UTC m=+39.887777443 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/000696c7-5f3f-469b-a8bd-3df2d07590b4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v9q8v" (UID: "000696c7-5f3f-469b-a8bd-3df2d07590b4") : configmap "kube-rbac-proxy" not found
Sep 11 00:58:43.490724 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:43.490675    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/000696c7-5f3f-469b-a8bd-3df2d07590b4-insights-runtime-extractor-tls podName:000696c7-5f3f-469b-a8bd-3df2d07590b4 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:44.490656586 +0000 UTC m=+39.887848737 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/000696c7-5f3f-469b-a8bd-3df2d07590b4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v9q8v" (UID: "000696c7-5f3f-469b-a8bd-3df2d07590b4") : secret "insights-runtime-extractor-tls" not found
Sep 11 00:58:43.573151 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:43.573105    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:43.583898 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:43.583859    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-cs2xq" podStartSLOduration=2.3852337439999998 podStartE2EDuration="4.583842658s" totalImagesPullingTime="2.198608914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:58:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:58:43.507971782 +0000 UTC m=+38.905163954" watchObservedRunningTime="2026-09-11 00:58:43.583842658 +0000 UTC m=+38.981034827"
Sep 11 00:58:43.584157 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:43.584143    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-hj6rz"]
Sep 11 00:58:43.598070 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:43.598053    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-hj6rz"
Sep 11 00:58:43.599922 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:43.599865    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-hj6rz"]
Sep 11 00:58:43.601068 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:43.601042    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 11 00:58:43.601166 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:43.601081    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-tsp68\""
Sep 11 00:58:43.601358 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:43.601339    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 11 00:58:43.645205 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:43.645181    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:43.692454 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:43.692432    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7f56q\" (UniqueName: \"kubernetes.io/projected/8fd16e7e-9fe7-499d-becf-de3ba31c8549-kube-api-access-7f56q\") pod \"migrator-8b4d697c5-hj6rz\" (UID: \"8fd16e7e-9fe7-499d-becf-de3ba31c8549\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-hj6rz"
Sep 11 00:58:43.745557 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:43.745484    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:43.793727 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:43.793688    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f56q\" (UniqueName: \"kubernetes.io/projected/8fd16e7e-9fe7-499d-becf-de3ba31c8549-kube-api-access-7f56q\") pod \"migrator-8b4d697c5-hj6rz\" (UID: \"8fd16e7e-9fe7-499d-becf-de3ba31c8549\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-hj6rz"
Sep 11 00:58:43.802064 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:43.802033    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7f56q\" (UniqueName: \"kubernetes.io/projected/8fd16e7e-9fe7-499d-becf-de3ba31c8549-kube-api-access-7f56q\") pod \"migrator-8b4d697c5-hj6rz\" (UID: \"8fd16e7e-9fe7-499d-becf-de3ba31c8549\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-hj6rz"
Sep 11 00:58:43.907783 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:43.907761    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-hj6rz"
Sep 11 00:58:44.205030 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:44.205003    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-hj6rz"]
Sep 11 00:58:44.208890 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:44.208864    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fd16e7e_9fe7_499d_becf_de3ba31c8549.slice/crio-9ddf4b8ce0245f7f0a40a49d7f607cebcf4a2865dbcd9522d1147df12e9178c2 WatchSource:0}: Error finding container 9ddf4b8ce0245f7f0a40a49d7f607cebcf4a2865dbcd9522d1147df12e9178c2: Status 404 returned error can't find the container with id 9ddf4b8ce0245f7f0a40a49d7f607cebcf4a2865dbcd9522d1147df12e9178c2
Sep 11 00:58:44.467526 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:44.467448    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-hj6rz" event={"ID":"8fd16e7e-9fe7-499d-becf-de3ba31c8549","Type":"ContainerStarted","Data":"9ddf4b8ce0245f7f0a40a49d7f607cebcf4a2865dbcd9522d1147df12e9178c2"}
Sep 11 00:58:44.499176 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:44.499148    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/000696c7-5f3f-469b-a8bd-3df2d07590b4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v9q8v\" (UID: \"000696c7-5f3f-469b-a8bd-3df2d07590b4\") " pod="openshift-insights/insights-runtime-extractor-v9q8v"
Sep 11 00:58:44.499299 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:44.499189    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/000696c7-5f3f-469b-a8bd-3df2d07590b4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v9q8v\" (UID: \"000696c7-5f3f-469b-a8bd-3df2d07590b4\") " pod="openshift-insights/insights-runtime-extractor-v9q8v"
Sep 11 00:58:44.499299 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:44.499278    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 00:58:44.499299 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:44.499291    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 00:58:44.499422 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:44.499332    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/000696c7-5f3f-469b-a8bd-3df2d07590b4-insights-runtime-extractor-tls podName:000696c7-5f3f-469b-a8bd-3df2d07590b4 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:46.499318527 +0000 UTC m=+41.896510681 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/000696c7-5f3f-469b-a8bd-3df2d07590b4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v9q8v" (UID: "000696c7-5f3f-469b-a8bd-3df2d07590b4") : secret "insights-runtime-extractor-tls" not found
Sep 11 00:58:44.499422 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:44.499345    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/000696c7-5f3f-469b-a8bd-3df2d07590b4-kube-rbac-proxy-cm podName:000696c7-5f3f-469b-a8bd-3df2d07590b4 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:46.499339213 +0000 UTC m=+41.896531364 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/000696c7-5f3f-469b-a8bd-3df2d07590b4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v9q8v" (UID: "000696c7-5f3f-469b-a8bd-3df2d07590b4") : configmap "kube-rbac-proxy" not found
Sep 11 00:58:44.817269 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:44.817187    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-lsp5w"]
Sep 11 00:58:44.837647 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:44.837617    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-lsp5w"]
Sep 11 00:58:44.837793 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:44.837726    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-lsp5w"
Sep 11 00:58:44.840839 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:44.840820    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 11 00:58:44.840982 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:44.840940    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 11 00:58:44.841523 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:44.841500    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-p6r65\""
Sep 11 00:58:44.841757 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:44.841741    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 11 00:58:44.841900 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:44.841747    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 11 00:58:44.927450 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:44.927427    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:44.929294 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:44.929279    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zw22q_d097ee33-8900-43b7-9eae-35ac88373d1d/dns-node-resolver/0.log"
Sep 11 00:58:45.003192 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:45.003158    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a37cc67b-0963-428d-87c4-286f3b3cb07e-signing-cabundle\") pod \"service-ca-568869fdc5-lsp5w\" (UID: \"a37cc67b-0963-428d-87c4-286f3b3cb07e\") " pod="openshift-service-ca/service-ca-568869fdc5-lsp5w"
Sep 11 00:58:45.003339 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:45.003198    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a37cc67b-0963-428d-87c4-286f3b3cb07e-signing-key\") pod \"service-ca-568869fdc5-lsp5w\" (UID: \"a37cc67b-0963-428d-87c4-286f3b3cb07e\") " pod="openshift-service-ca/service-ca-568869fdc5-lsp5w"
Sep 11 00:58:45.003339 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:45.003301    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hwz9\" (UniqueName: \"kubernetes.io/projected/a37cc67b-0963-428d-87c4-286f3b3cb07e-kube-api-access-6hwz9\") pod \"service-ca-568869fdc5-lsp5w\" (UID: \"a37cc67b-0963-428d-87c4-286f3b3cb07e\") " pod="openshift-service-ca/service-ca-568869fdc5-lsp5w"
Sep 11 00:58:45.027144 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:45.027116    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:45.103617 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:45.103540    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hwz9\" (UniqueName: \"kubernetes.io/projected/a37cc67b-0963-428d-87c4-286f3b3cb07e-kube-api-access-6hwz9\") pod \"service-ca-568869fdc5-lsp5w\" (UID: \"a37cc67b-0963-428d-87c4-286f3b3cb07e\") " pod="openshift-service-ca/service-ca-568869fdc5-lsp5w"
Sep 11 00:58:45.103617 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:45.103604    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a37cc67b-0963-428d-87c4-286f3b3cb07e-signing-cabundle\") pod \"service-ca-568869fdc5-lsp5w\" (UID: \"a37cc67b-0963-428d-87c4-286f3b3cb07e\") " pod="openshift-service-ca/service-ca-568869fdc5-lsp5w"
Sep 11 00:58:45.103819 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:45.103632    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a37cc67b-0963-428d-87c4-286f3b3cb07e-signing-key\") pod \"service-ca-568869fdc5-lsp5w\" (UID: \"a37cc67b-0963-428d-87c4-286f3b3cb07e\") " pod="openshift-service-ca/service-ca-568869fdc5-lsp5w"
Sep 11 00:58:45.104385 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:45.104359    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a37cc67b-0963-428d-87c4-286f3b3cb07e-signing-cabundle\") pod \"service-ca-568869fdc5-lsp5w\" (UID: \"a37cc67b-0963-428d-87c4-286f3b3cb07e\") " pod="openshift-service-ca/service-ca-568869fdc5-lsp5w"
Sep 11 00:58:45.107283 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:45.107256    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a37cc67b-0963-428d-87c4-286f3b3cb07e-signing-key\") pod \"service-ca-568869fdc5-lsp5w\" (UID: \"a37cc67b-0963-428d-87c4-286f3b3cb07e\") " pod="openshift-service-ca/service-ca-568869fdc5-lsp5w"
Sep 11 00:58:45.113365 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:45.113344    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hwz9\" (UniqueName: \"kubernetes.io/projected/a37cc67b-0963-428d-87c4-286f3b3cb07e-kube-api-access-6hwz9\") pod \"service-ca-568869fdc5-lsp5w\" (UID: \"a37cc67b-0963-428d-87c4-286f3b3cb07e\") " pod="openshift-service-ca/service-ca-568869fdc5-lsp5w"
Sep 11 00:58:45.149097 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:45.149071    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-lsp5w"
Sep 11 00:58:45.306090 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:45.306062    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:45.362591 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:45.362536    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:45.514437 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:45.514416    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-lsp5w"]
Sep 11 00:58:45.517453 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:45.517430    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda37cc67b_0963_428d_87c4_286f3b3cb07e.slice/crio-7055d8b350b52a5ae00f33e92e560d984fc26ab341ec38e8fbd420a8182ef1a5 WatchSource:0}: Error finding container 7055d8b350b52a5ae00f33e92e560d984fc26ab341ec38e8fbd420a8182ef1a5: Status 404 returned error can't find the container with id 7055d8b350b52a5ae00f33e92e560d984fc26ab341ec38e8fbd420a8182ef1a5
Sep 11 00:58:45.730025 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:45.730006    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hh5wm_1625cd1f-5711-4d05-9cfc-74a2a2c1c637/node-ca/0.log"
Sep 11 00:58:45.973215 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:45.973183    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:46.330693 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:46.330605    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-cs2xq_af240b86-c806-4cf9-a3f0-0bee21e7abfe/serve-healthcheck-canary/0.log"
Sep 11 00:58:46.473881 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:46.473845    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-lsp5w" event={"ID":"a37cc67b-0963-428d-87c4-286f3b3cb07e","Type":"ContainerStarted","Data":"7055d8b350b52a5ae00f33e92e560d984fc26ab341ec38e8fbd420a8182ef1a5"}
Sep 11 00:58:46.475812 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:46.475790    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-hj6rz" event={"ID":"8fd16e7e-9fe7-499d-becf-de3ba31c8549","Type":"ContainerStarted","Data":"dfb1a9ba677af20d76b3b01080e3e92bb8e240e752bac90e036fed5941082c49"}
Sep 11 00:58:46.475950 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:46.475832    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-hj6rz" event={"ID":"8fd16e7e-9fe7-499d-becf-de3ba31c8549","Type":"ContainerStarted","Data":"5d3664c3ba21949d4c08129e55ebf0da0b79098013d6f9fd83f50ec078f501be"}
Sep 11 00:58:46.494624 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:46.494579    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-hj6rz" podStartSLOduration=2.198721359 podStartE2EDuration="3.49456184s" totalImagesPullingTime="1.295840481s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:58:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:58:46.494172263 +0000 UTC m=+41.891364436" watchObservedRunningTime="2026-09-11 00:58:46.49456184 +0000 UTC m=+41.891754012"
Sep 11 00:58:46.514518 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:46.514474    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/000696c7-5f3f-469b-a8bd-3df2d07590b4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v9q8v\" (UID: \"000696c7-5f3f-469b-a8bd-3df2d07590b4\") " pod="openshift-insights/insights-runtime-extractor-v9q8v"
Sep 11 00:58:46.514928 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:46.514539    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/000696c7-5f3f-469b-a8bd-3df2d07590b4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v9q8v\" (UID: \"000696c7-5f3f-469b-a8bd-3df2d07590b4\") " pod="openshift-insights/insights-runtime-extractor-v9q8v"
Sep 11 00:58:46.514928 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:46.514613    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 00:58:46.514928 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:46.514662    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 00:58:46.514928 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:46.514686    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/000696c7-5f3f-469b-a8bd-3df2d07590b4-insights-runtime-extractor-tls podName:000696c7-5f3f-469b-a8bd-3df2d07590b4 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:50.514661722 +0000 UTC m=+45.911853875 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/000696c7-5f3f-469b-a8bd-3df2d07590b4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v9q8v" (UID: "000696c7-5f3f-469b-a8bd-3df2d07590b4") : secret "insights-runtime-extractor-tls" not found
Sep 11 00:58:46.514928 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:46.514705    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/000696c7-5f3f-469b-a8bd-3df2d07590b4-kube-rbac-proxy-cm podName:000696c7-5f3f-469b-a8bd-3df2d07590b4 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:50.514694355 +0000 UTC m=+45.911886506 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/000696c7-5f3f-469b-a8bd-3df2d07590b4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v9q8v" (UID: "000696c7-5f3f-469b-a8bd-3df2d07590b4") : configmap "kube-rbac-proxy" not found
Sep 11 00:58:47.321560 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:47.321529    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dda2c7fa-a50c-483c-8709-775d04d924c2-metrics-tls\") pod \"dns-default-j9mpz\" (UID: \"dda2c7fa-a50c-483c-8709-775d04d924c2\") " pod="openshift-dns/dns-default-j9mpz"
Sep 11 00:58:47.321717 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:47.321672    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 00:58:47.321759 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:47.321736    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dda2c7fa-a50c-483c-8709-775d04d924c2-metrics-tls podName:dda2c7fa-a50c-483c-8709-775d04d924c2 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:55.321722115 +0000 UTC m=+50.718914265 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dda2c7fa-a50c-483c-8709-775d04d924c2-metrics-tls") pod "dns-default-j9mpz" (UID: "dda2c7fa-a50c-483c-8709-775d04d924c2") : secret "dns-default-metrics-tls" not found
Sep 11 00:58:48.481926 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:48.481879    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-lsp5w" event={"ID":"a37cc67b-0963-428d-87c4-286f3b3cb07e","Type":"ContainerStarted","Data":"1736538a0d9a8f56e1f9000401da95cb67ce34965e1b03b2e440be7d65d87e46"}
Sep 11 00:58:48.500538 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:48.500485    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-lsp5w" podStartSLOduration=2.489723852 podStartE2EDuration="4.50046716s" totalImagesPullingTime="2.010743308s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:58:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:58:48.500230391 +0000 UTC m=+43.897422570" watchObservedRunningTime="2026-09-11 00:58:48.50046716 +0000 UTC m=+43.897659333"
Sep 11 00:58:48.840719 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:48.840647    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:48.967353 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:48.967315    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:49.084117 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:49.084078    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:49.183624 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:49.183591    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:49.286704 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:49.286649    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:49.399012 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:49.398978    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:49.637272 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:49.637179    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:50.431099 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:50.431065    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:50.548836 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:50.548803    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/000696c7-5f3f-469b-a8bd-3df2d07590b4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v9q8v\" (UID: \"000696c7-5f3f-469b-a8bd-3df2d07590b4\") " pod="openshift-insights/insights-runtime-extractor-v9q8v"
Sep 11 00:58:50.549037 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:50.548844    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/000696c7-5f3f-469b-a8bd-3df2d07590b4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v9q8v\" (UID: \"000696c7-5f3f-469b-a8bd-3df2d07590b4\") " pod="openshift-insights/insights-runtime-extractor-v9q8v"
Sep 11 00:58:50.549037 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:50.548940    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 00:58:50.549037 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:50.548991    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/000696c7-5f3f-469b-a8bd-3df2d07590b4-kube-rbac-proxy-cm podName:000696c7-5f3f-469b-a8bd-3df2d07590b4 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:58.548977333 +0000 UTC m=+53.946169484 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/000696c7-5f3f-469b-a8bd-3df2d07590b4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v9q8v" (UID: "000696c7-5f3f-469b-a8bd-3df2d07590b4") : configmap "kube-rbac-proxy" not found
Sep 11 00:58:50.549037 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:50.549004    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 00:58:50.549198 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:50.549063    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/000696c7-5f3f-469b-a8bd-3df2d07590b4-insights-runtime-extractor-tls podName:000696c7-5f3f-469b-a8bd-3df2d07590b4 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:58.549046318 +0000 UTC m=+53.946238486 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/000696c7-5f3f-469b-a8bd-3df2d07590b4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v9q8v" (UID: "000696c7-5f3f-469b-a8bd-3df2d07590b4") : secret "insights-runtime-extractor-tls" not found
Sep 11 00:58:50.835148 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:50.835064    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:51.229295 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:51.229266    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:51.429704 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:51.429668    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:51.833464 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:51.833433    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:52.024503 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:52.024471    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:52.226940 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:52.226891    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:52.819567 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:52.819536    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:52.876011 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:52.875982    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:53.429963 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:53.429935    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:53.635360 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:53.635323    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:53.693508 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:53.693442    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:54.022842 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:54.022766    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:54.224643 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:54.224610    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:54.634291 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:54.634259    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:55.228551 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:55.228519    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:55.384104 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:55.384074    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dda2c7fa-a50c-483c-8709-775d04d924c2-metrics-tls\") pod \"dns-default-j9mpz\" (UID: \"dda2c7fa-a50c-483c-8709-775d04d924c2\") " pod="openshift-dns/dns-default-j9mpz"
Sep 11 00:58:55.386854 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:55.386826    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dda2c7fa-a50c-483c-8709-775d04d924c2-metrics-tls\") pod \"dns-default-j9mpz\" (UID: \"dda2c7fa-a50c-483c-8709-775d04d924c2\") " pod="openshift-dns/dns-default-j9mpz"
Sep 11 00:58:55.410159 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:55.410131    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:55.414069 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:55.414055    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-j9mpz"
Sep 11 00:58:55.691047 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:55.691013    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-j9mpz"]
Sep 11 00:58:55.695131 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:58:55.695088    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddda2c7fa_a50c_483c_8709_775d04d924c2.slice/crio-29569910e54d692a3c83463a013371cbc3cdb59c31e06806f69c9edac6426bb7 WatchSource:0}: Error finding container 29569910e54d692a3c83463a013371cbc3cdb59c31e06806f69c9edac6426bb7: Status 404 returned error can't find the container with id 29569910e54d692a3c83463a013371cbc3cdb59c31e06806f69c9edac6426bb7
Sep 11 00:58:55.822115 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:55.822088    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:56.029297 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:56.029219    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:56.224088 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:56.224039    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:56.324238 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:56.324163    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:56.505286 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:56.505247    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-j9mpz" event={"ID":"dda2c7fa-a50c-483c-8709-775d04d924c2","Type":"ContainerStarted","Data":"29569910e54d692a3c83463a013371cbc3cdb59c31e06806f69c9edac6426bb7"}
Sep 11 00:58:56.627721 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:56.627647    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:57.230929 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:57.230880    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:57.520129 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:57.520065    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:57.622079 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:57.622040    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:58.024967 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:58.024936    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:58.168459 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:58.168428    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:58.269393 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:58.269363    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:58.340211 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:58.340188    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:58.432134 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:58.432083    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:58.494684 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:58.494658    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:58.513322 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:58.513300    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-j9mpz" event={"ID":"dda2c7fa-a50c-483c-8709-775d04d924c2","Type":"ContainerStarted","Data":"2babe36ed4fc7440636212e1e05d97f29d86efe5450629ab83b70363c4612703"}
Sep 11 00:58:58.513406 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:58.513342    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-j9mpz" event={"ID":"dda2c7fa-a50c-483c-8709-775d04d924c2","Type":"ContainerStarted","Data":"873b953b79ec661bf07708352db8765a2fad404f04babb0f2aa3ea9986748aec"}
Sep 11 00:58:58.513453 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:58.513436    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-j9mpz"
Sep 11 00:58:58.532614 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:58.532576    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-j9mpz" podStartSLOduration=17.57819289 podStartE2EDuration="19.532563213s" totalImagesPullingTime="1.954370323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:58:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:58:58.531359689 +0000 UTC m=+53.928551862" watchObservedRunningTime="2026-09-11 00:58:58.532563213 +0000 UTC m=+53.929755386"
Sep 11 00:58:58.609617 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:58.609588    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/000696c7-5f3f-469b-a8bd-3df2d07590b4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v9q8v\" (UID: \"000696c7-5f3f-469b-a8bd-3df2d07590b4\") " pod="openshift-insights/insights-runtime-extractor-v9q8v"
Sep 11 00:58:58.609729 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:58.609621    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/000696c7-5f3f-469b-a8bd-3df2d07590b4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v9q8v\" (UID: \"000696c7-5f3f-469b-a8bd-3df2d07590b4\") " pod="openshift-insights/insights-runtime-extractor-v9q8v"
Sep 11 00:58:58.609794 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:58.609766    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 00:58:58.609845 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:58:58.609836    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/000696c7-5f3f-469b-a8bd-3df2d07590b4-insights-runtime-extractor-tls podName:000696c7-5f3f-469b-a8bd-3df2d07590b4 nodeName:}" failed. No retries permitted until 2026-09-11 00:59:14.609817122 +0000 UTC m=+70.007009278 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/000696c7-5f3f-469b-a8bd-3df2d07590b4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v9q8v" (UID: "000696c7-5f3f-469b-a8bd-3df2d07590b4") : secret "insights-runtime-extractor-tls" not found
Sep 11 00:58:58.610252 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:58.610229    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/000696c7-5f3f-469b-a8bd-3df2d07590b4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v9q8v\" (UID: \"000696c7-5f3f-469b-a8bd-3df2d07590b4\") " pod="openshift-insights/insights-runtime-extractor-v9q8v"
Sep 11 00:58:58.634542 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:58.634515    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:58.691428 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:58.691360    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:58.824801 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:58.824746    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:59.030705 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:59.030632    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:59.088854 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:59.088835    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:59.243575 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:59.243520    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:59.426559 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:59.426532    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:59.628319 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:59.628291    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:59.685373 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:59.685310    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:58:59.861964 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:58:59.861935    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:00.477701 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:00.477672    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:00.874804 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:00.874732    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:02.215491 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:02.215461    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:02.310237 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:02.310207    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:02.409984 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:02.409945    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:02.721428 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:02.721388    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:04.438237 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:04.438213    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d7crj"
Sep 11 00:59:05.218107 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.218077    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:05.223811 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.223787    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-84ccd59f5-rz4qk"]
Sep 11 00:59:05.254961 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.254935    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84ccd59f5-rz4qk"]
Sep 11 00:59:05.255056 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.255027    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84ccd59f5-rz4qk"
Sep 11 00:59:05.257390 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.257370    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hstjm\""
Sep 11 00:59:05.257510 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.257493    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 11 00:59:05.258112 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.258095    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 11 00:59:05.258807 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.258787    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 11 00:59:05.263714 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.263696    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 11 00:59:05.308775 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.308736    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:05.360194 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.360170    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-db4jh\" (UniqueName: \"kubernetes.io/projected/edba57c7-d190-4a32-8614-2b18ed243970-kube-api-access-db4jh\") pod \"image-registry-84ccd59f5-rz4qk\" (UID: \"edba57c7-d190-4a32-8614-2b18ed243970\") " pod="openshift-image-registry/image-registry-84ccd59f5-rz4qk"
Sep 11 00:59:05.360303 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.360200    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edba57c7-d190-4a32-8614-2b18ed243970-registry-tls\") pod \"image-registry-84ccd59f5-rz4qk\" (UID: \"edba57c7-d190-4a32-8614-2b18ed243970\") " pod="openshift-image-registry/image-registry-84ccd59f5-rz4qk"
Sep 11 00:59:05.360303 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.360219    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/edba57c7-d190-4a32-8614-2b18ed243970-installation-pull-secrets\") pod \"image-registry-84ccd59f5-rz4qk\" (UID: \"edba57c7-d190-4a32-8614-2b18ed243970\") " pod="openshift-image-registry/image-registry-84ccd59f5-rz4qk"
Sep 11 00:59:05.360303 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.360256    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/edba57c7-d190-4a32-8614-2b18ed243970-ca-trust-extracted\") pod \"image-registry-84ccd59f5-rz4qk\" (UID: \"edba57c7-d190-4a32-8614-2b18ed243970\") " pod="openshift-image-registry/image-registry-84ccd59f5-rz4qk"
Sep 11 00:59:05.360413 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.360304    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/edba57c7-d190-4a32-8614-2b18ed243970-trusted-ca\") pod \"image-registry-84ccd59f5-rz4qk\" (UID: \"edba57c7-d190-4a32-8614-2b18ed243970\") " pod="openshift-image-registry/image-registry-84ccd59f5-rz4qk"
Sep 11 00:59:05.360413 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.360350    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/edba57c7-d190-4a32-8614-2b18ed243970-bound-sa-token\") pod \"image-registry-84ccd59f5-rz4qk\" (UID: \"edba57c7-d190-4a32-8614-2b18ed243970\") " pod="openshift-image-registry/image-registry-84ccd59f5-rz4qk"
Sep 11 00:59:05.360413 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.360392    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/edba57c7-d190-4a32-8614-2b18ed243970-image-registry-private-configuration\") pod \"image-registry-84ccd59f5-rz4qk\" (UID: \"edba57c7-d190-4a32-8614-2b18ed243970\") " pod="openshift-image-registry/image-registry-84ccd59f5-rz4qk"
Sep 11 00:59:05.360505 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.360433    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/edba57c7-d190-4a32-8614-2b18ed243970-registry-certificates\") pod \"image-registry-84ccd59f5-rz4qk\" (UID: \"edba57c7-d190-4a32-8614-2b18ed243970\") " pod="openshift-image-registry/image-registry-84ccd59f5-rz4qk"
Sep 11 00:59:05.402190 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.402140    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:05.461320 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.461296    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/edba57c7-d190-4a32-8614-2b18ed243970-trusted-ca\") pod \"image-registry-84ccd59f5-rz4qk\" (UID: \"edba57c7-d190-4a32-8614-2b18ed243970\") " pod="openshift-image-registry/image-registry-84ccd59f5-rz4qk"
Sep 11 00:59:05.461691 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.461323    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/edba57c7-d190-4a32-8614-2b18ed243970-bound-sa-token\") pod \"image-registry-84ccd59f5-rz4qk\" (UID: \"edba57c7-d190-4a32-8614-2b18ed243970\") " pod="openshift-image-registry/image-registry-84ccd59f5-rz4qk"
Sep 11 00:59:05.461691 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.461347    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/edba57c7-d190-4a32-8614-2b18ed243970-image-registry-private-configuration\") pod \"image-registry-84ccd59f5-rz4qk\" (UID: \"edba57c7-d190-4a32-8614-2b18ed243970\") " pod="openshift-image-registry/image-registry-84ccd59f5-rz4qk"
Sep 11 00:59:05.461691 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.461399    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/edba57c7-d190-4a32-8614-2b18ed243970-registry-certificates\") pod \"image-registry-84ccd59f5-rz4qk\" (UID: \"edba57c7-d190-4a32-8614-2b18ed243970\") " pod="openshift-image-registry/image-registry-84ccd59f5-rz4qk"
Sep 11 00:59:05.461691 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.461421    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-db4jh\" (UniqueName: \"kubernetes.io/projected/edba57c7-d190-4a32-8614-2b18ed243970-kube-api-access-db4jh\") pod \"image-registry-84ccd59f5-rz4qk\" (UID: \"edba57c7-d190-4a32-8614-2b18ed243970\") " pod="openshift-image-registry/image-registry-84ccd59f5-rz4qk"
Sep 11 00:59:05.461691 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.461447    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edba57c7-d190-4a32-8614-2b18ed243970-registry-tls\") pod \"image-registry-84ccd59f5-rz4qk\" (UID: \"edba57c7-d190-4a32-8614-2b18ed243970\") " pod="openshift-image-registry/image-registry-84ccd59f5-rz4qk"
Sep 11 00:59:05.461691 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.461468    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/edba57c7-d190-4a32-8614-2b18ed243970-installation-pull-secrets\") pod \"image-registry-84ccd59f5-rz4qk\" (UID: \"edba57c7-d190-4a32-8614-2b18ed243970\") " pod="openshift-image-registry/image-registry-84ccd59f5-rz4qk"
Sep 11 00:59:05.462003 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.461704    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/edba57c7-d190-4a32-8614-2b18ed243970-ca-trust-extracted\") pod \"image-registry-84ccd59f5-rz4qk\" (UID: \"edba57c7-d190-4a32-8614-2b18ed243970\") " pod="openshift-image-registry/image-registry-84ccd59f5-rz4qk"
Sep 11 00:59:05.462264 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.462188    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/edba57c7-d190-4a32-8614-2b18ed243970-ca-trust-extracted\") pod \"image-registry-84ccd59f5-rz4qk\" (UID: \"edba57c7-d190-4a32-8614-2b18ed243970\") " pod="openshift-image-registry/image-registry-84ccd59f5-rz4qk"
Sep 11 00:59:05.462707 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.462682    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/edba57c7-d190-4a32-8614-2b18ed243970-registry-certificates\") pod \"image-registry-84ccd59f5-rz4qk\" (UID: \"edba57c7-d190-4a32-8614-2b18ed243970\") " pod="openshift-image-registry/image-registry-84ccd59f5-rz4qk"
Sep 11 00:59:05.463017 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.462997    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/edba57c7-d190-4a32-8614-2b18ed243970-trusted-ca\") pod \"image-registry-84ccd59f5-rz4qk\" (UID: \"edba57c7-d190-4a32-8614-2b18ed243970\") " pod="openshift-image-registry/image-registry-84ccd59f5-rz4qk"
Sep 11 00:59:05.464207 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.464175    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edba57c7-d190-4a32-8614-2b18ed243970-registry-tls\") pod \"image-registry-84ccd59f5-rz4qk\" (UID: \"edba57c7-d190-4a32-8614-2b18ed243970\") " pod="openshift-image-registry/image-registry-84ccd59f5-rz4qk"
Sep 11 00:59:05.464308 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.464205    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/edba57c7-d190-4a32-8614-2b18ed243970-image-registry-private-configuration\") pod \"image-registry-84ccd59f5-rz4qk\" (UID: \"edba57c7-d190-4a32-8614-2b18ed243970\") " pod="openshift-image-registry/image-registry-84ccd59f5-rz4qk"
Sep 11 00:59:05.464368 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.464334    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/edba57c7-d190-4a32-8614-2b18ed243970-installation-pull-secrets\") pod \"image-registry-84ccd59f5-rz4qk\" (UID: \"edba57c7-d190-4a32-8614-2b18ed243970\") " pod="openshift-image-registry/image-registry-84ccd59f5-rz4qk"
Sep 11 00:59:05.469199 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.469136    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-db4jh\" (UniqueName: \"kubernetes.io/projected/edba57c7-d190-4a32-8614-2b18ed243970-kube-api-access-db4jh\") pod \"image-registry-84ccd59f5-rz4qk\" (UID: \"edba57c7-d190-4a32-8614-2b18ed243970\") " pod="openshift-image-registry/image-registry-84ccd59f5-rz4qk"
Sep 11 00:59:05.469294 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.469260    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/edba57c7-d190-4a32-8614-2b18ed243970-bound-sa-token\") pod \"image-registry-84ccd59f5-rz4qk\" (UID: \"edba57c7-d190-4a32-8614-2b18ed243970\") " pod="openshift-image-registry/image-registry-84ccd59f5-rz4qk"
Sep 11 00:59:05.523946 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.523897    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:05.563656 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.563633    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84ccd59f5-rz4qk"
Sep 11 00:59:05.583252 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.583231    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:05.696517 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.696493    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:05.796574 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.796486    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:05.861113 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:05.861089    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84ccd59f5-rz4qk"]
Sep 11 00:59:05.864024 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:59:05.864002    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedba57c7_d190_4a32_8614_2b18ed243970.slice/crio-b2f10169a377e5b6e88ff9ade3d3690650674e5c6508727755a67bec36d91711 WatchSource:0}: Error finding container b2f10169a377e5b6e88ff9ade3d3690650674e5c6508727755a67bec36d91711: Status 404 returned error can't find the container with id b2f10169a377e5b6e88ff9ade3d3690650674e5c6508727755a67bec36d91711
Sep 11 00:59:06.542255 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:06.542217    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84ccd59f5-rz4qk" event={"ID":"edba57c7-d190-4a32-8614-2b18ed243970","Type":"ContainerStarted","Data":"31909c6841581461e981d245bdc2f160909f116f1e3466bf23c13386cf1f0710"}
Sep 11 00:59:06.542871 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:06.542291    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84ccd59f5-rz4qk" event={"ID":"edba57c7-d190-4a32-8614-2b18ed243970","Type":"ContainerStarted","Data":"b2f10169a377e5b6e88ff9ade3d3690650674e5c6508727755a67bec36d91711"}
Sep 11 00:59:06.542871 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:06.542445    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-84ccd59f5-rz4qk"
Sep 11 00:59:06.561811 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:06.561752    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-84ccd59f5-rz4qk" podStartSLOduration=1.561740637 podStartE2EDuration="1.561740637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:59:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:59:06.56072711 +0000 UTC m=+61.957919282" watchObservedRunningTime="2026-09-11 00:59:06.561740637 +0000 UTC m=+61.958932809"
Sep 11 00:59:08.395106 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:08.395068    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:08.484064 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:08.484028    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:08.521700 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:08.521670    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-j9mpz"
Sep 11 00:59:08.554891 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:08.554858    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:08.612795 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:08.612769    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:08.716868 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:08.716841    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:09.587541 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:09.587507    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:09.687824 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:09.687797    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:09.898032 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:09.897953    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff3ee2a6-e101-4cde-8d89-489f1db81ab8-original-pull-secret\") pod \"global-pull-secret-syncer-jbpsj\" (UID: \"ff3ee2a6-e101-4cde-8d89-489f1db81ab8\") " pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:59:09.898032 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:09.897998    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77093864-53b6-4044-952a-d8f1a965dd1d-metrics-certs\") pod \"network-metrics-daemon-9s5p9\" (UID: \"77093864-53b6-4044-952a-d8f1a965dd1d\") " pod="openshift-multus/network-metrics-daemon-9s5p9"
Sep 11 00:59:09.900354 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:09.900335    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 00:59:09.900949 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:09.900923    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 00:59:09.910690 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:09.910673    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff3ee2a6-e101-4cde-8d89-489f1db81ab8-original-pull-secret\") pod \"global-pull-secret-syncer-jbpsj\" (UID: \"ff3ee2a6-e101-4cde-8d89-489f1db81ab8\") " pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:59:09.910805 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:09.910784    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77093864-53b6-4044-952a-d8f1a965dd1d-metrics-certs\") pod \"network-metrics-daemon-9s5p9\" (UID: \"77093864-53b6-4044-952a-d8f1a965dd1d\") " pod="openshift-multus/network-metrics-daemon-9s5p9"
Sep 11 00:59:09.999176 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:09.999141    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tdws\" (UniqueName: \"kubernetes.io/projected/526304d7-8bd3-4196-a695-f5845e8846a1-kube-api-access-2tdws\") pod \"network-check-target-9hl4x\" (UID: \"526304d7-8bd3-4196-a695-f5845e8846a1\") " pod="openshift-network-diagnostics/network-check-target-9hl4x"
Sep 11 00:59:10.001492 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:10.001476    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 00:59:10.012057 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:10.012037    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 00:59:10.023028 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:10.023004    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tdws\" (UniqueName: \"kubernetes.io/projected/526304d7-8bd3-4196-a695-f5845e8846a1-kube-api-access-2tdws\") pod \"network-check-target-9hl4x\" (UID: \"526304d7-8bd3-4196-a695-f5845e8846a1\") " pod="openshift-network-diagnostics/network-check-target-9hl4x"
Sep 11 00:59:10.107236 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:10.107204    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbpsj"
Sep 11 00:59:10.118297 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:10.118270    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zpsrm\""
Sep 11 00:59:10.123158 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:10.123139    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-h86gq\""
Sep 11 00:59:10.126888 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:10.126871    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9s5p9"
Sep 11 00:59:10.131592 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:10.131575    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hl4x"
Sep 11 00:59:10.377697 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:10.377662    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jbpsj"]
Sep 11 00:59:10.381973 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:59:10.380341    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff3ee2a6_e101_4cde_8d89_489f1db81ab8.slice/crio-4b70277edd25d9f4b2ea45aafab0ab2ab5e80a5b0355a84502458b0dd50aa016 WatchSource:0}: Error finding container 4b70277edd25d9f4b2ea45aafab0ab2ab5e80a5b0355a84502458b0dd50aa016: Status 404 returned error can't find the container with id 4b70277edd25d9f4b2ea45aafab0ab2ab5e80a5b0355a84502458b0dd50aa016
Sep 11 00:59:10.403200 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:10.403177    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9s5p9"]
Sep 11 00:59:10.405974 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:59:10.405899    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77093864_53b6_4044_952a_d8f1a965dd1d.slice/crio-3998131b5db8bbc0881f4d7220fe4583590fb70cb2f3d3291d95e618e5c2bd4a WatchSource:0}: Error finding container 3998131b5db8bbc0881f4d7220fe4583590fb70cb2f3d3291d95e618e5c2bd4a: Status 404 returned error can't find the container with id 3998131b5db8bbc0881f4d7220fe4583590fb70cb2f3d3291d95e618e5c2bd4a
Sep 11 00:59:10.406299 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:10.406279    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9hl4x"]
Sep 11 00:59:10.408876 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:59:10.408854    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod526304d7_8bd3_4196_a695_f5845e8846a1.slice/crio-39a0eed61e4f3ec6bb8c6915896e4c2bf8b5e10438cd35a4a15c86e97b0d52cd WatchSource:0}: Error finding container 39a0eed61e4f3ec6bb8c6915896e4c2bf8b5e10438cd35a4a15c86e97b0d52cd: Status 404 returned error can't find the container with id 39a0eed61e4f3ec6bb8c6915896e4c2bf8b5e10438cd35a4a15c86e97b0d52cd
Sep 11 00:59:10.554663 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:10.554626    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jbpsj" event={"ID":"ff3ee2a6-e101-4cde-8d89-489f1db81ab8","Type":"ContainerStarted","Data":"4b70277edd25d9f4b2ea45aafab0ab2ab5e80a5b0355a84502458b0dd50aa016"}
Sep 11 00:59:10.555611 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:10.555582    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9hl4x" event={"ID":"526304d7-8bd3-4196-a695-f5845e8846a1","Type":"ContainerStarted","Data":"39a0eed61e4f3ec6bb8c6915896e4c2bf8b5e10438cd35a4a15c86e97b0d52cd"}
Sep 11 00:59:10.556567 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:10.556544    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9s5p9" event={"ID":"77093864-53b6-4044-952a-d8f1a965dd1d","Type":"ContainerStarted","Data":"3998131b5db8bbc0881f4d7220fe4583590fb70cb2f3d3291d95e618e5c2bd4a"}
Sep 11 00:59:10.680145 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:10.680113    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:12.266321 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:12.266230    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:12.368309 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:12.368215    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:12.467955 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:12.467902    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:12.567665 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:12.567629    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9s5p9" event={"ID":"77093864-53b6-4044-952a-d8f1a965dd1d","Type":"ContainerStarted","Data":"d3ff5815afd2a605aef6c6ba5588f044372ca250a6b55ff954361131731f7147"}
Sep 11 00:59:12.567850 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:12.567742    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9s5p9" event={"ID":"77093864-53b6-4044-952a-d8f1a965dd1d","Type":"ContainerStarted","Data":"ae1e5281ad6f5877e6caece6a268d0a46c2959bc0dd2e54bda7fcdaf0d6568bd"}
Sep 11 00:59:12.592010 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:12.591953    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9s5p9" podStartSLOduration=66.177025542 podStartE2EDuration="1m7.591937024s" totalImagesPullingTime="1.414911482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:58:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:59:12.590965499 +0000 UTC m=+67.988157675" watchObservedRunningTime="2026-09-11 00:59:12.591937024 +0000 UTC m=+67.989129197"
Sep 11 00:59:12.684297 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:12.684221    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:12.781753 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:12.781721    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:13.277436 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:13.277365    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:13.336534 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:13.336497    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:14.633718 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:14.633684    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/000696c7-5f3f-469b-a8bd-3df2d07590b4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v9q8v\" (UID: \"000696c7-5f3f-469b-a8bd-3df2d07590b4\") " pod="openshift-insights/insights-runtime-extractor-v9q8v"
Sep 11 00:59:14.636426 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:14.636398    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/000696c7-5f3f-469b-a8bd-3df2d07590b4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v9q8v\" (UID: \"000696c7-5f3f-469b-a8bd-3df2d07590b4\") " pod="openshift-insights/insights-runtime-extractor-v9q8v"
Sep 11 00:59:14.891951 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:14.891863    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mtm2j\""
Sep 11 00:59:14.899741 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:14.899709    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-v9q8v"
Sep 11 00:59:15.222870 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:15.222819    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-v9q8v"]
Sep 11 00:59:15.582019 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:15.581899    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9hl4x" event={"ID":"526304d7-8bd3-4196-a695-f5845e8846a1","Type":"ContainerStarted","Data":"2a10ac5af8cc603b035536319f5e1cdd46ae3c2615b49679af5edd2a5a0d7d67"}
Sep 11 00:59:15.582175 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:15.582038    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9hl4x"
Sep 11 00:59:15.583393 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:15.583368    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jbpsj" event={"ID":"ff3ee2a6-e101-4cde-8d89-489f1db81ab8","Type":"ContainerStarted","Data":"7c4baa86740fcae7b653fca0c2283ea05e42b33ef8174dbfa9c7ee027dc02389"}
Sep 11 00:59:15.584818 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:15.584796    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v9q8v" event={"ID":"000696c7-5f3f-469b-a8bd-3df2d07590b4","Type":"ContainerStarted","Data":"5ec6d7cd4976485d9d6960d7dd260aa0e281d5e92f581007b07ffdcc9478e80f"}
Sep 11 00:59:15.584961 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:15.584841    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v9q8v" event={"ID":"000696c7-5f3f-469b-a8bd-3df2d07590b4","Type":"ContainerStarted","Data":"ab6f5879570e29c18f72ccc1769597de0bd893a64a784a68fd0f0245d6beef86"}
Sep 11 00:59:15.596149 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:15.596111    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9hl4x" podStartSLOduration=66.051250378 podStartE2EDuration="1m10.59609789s" totalImagesPullingTime="4.544847512s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:58:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:59:15.594844323 +0000 UTC m=+70.992036502" watchObservedRunningTime="2026-09-11 00:59:15.59609789 +0000 UTC m=+70.993290062"
Sep 11 00:59:15.610562 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:15.610495    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-jbpsj" podStartSLOduration=66.033292079 podStartE2EDuration="1m10.610479578s" totalImagesPullingTime="4.577187499s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 00:58:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:59:15.609993492 +0000 UTC m=+71.007185665" watchObservedRunningTime="2026-09-11 00:59:15.610479578 +0000 UTC m=+71.007671754"
Sep 11 00:59:15.910818 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:15.910787    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:16.591982 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:16.591940    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v9q8v" event={"ID":"000696c7-5f3f-469b-a8bd-3df2d07590b4","Type":"ContainerStarted","Data":"bc389f6bb9aa38615cb2bbc4eaf984c7f70b4217f50d9503b16b5e059df83a04"}
Sep 11 00:59:17.112866 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:17.112831    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:17.168106 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:17.168050    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:17.597267 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:17.597235    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v9q8v" event={"ID":"000696c7-5f3f-469b-a8bd-3df2d07590b4","Type":"ContainerStarted","Data":"b4f60ab3e1802b363e9b9951a4cf711f86300cc5c5cd1fef8fe3e2a2db107806"}
Sep 11 00:59:17.616142 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:17.616095    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-v9q8v" podStartSLOduration=33.744360225 podStartE2EDuration="35.616079887s" totalImagesPullingTime="1.871719662s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 00:58:42 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:59:17.615100296 +0000 UTC m=+73.012292481" watchObservedRunningTime="2026-09-11 00:59:17.616079887 +0000 UTC m=+73.013272057"
Sep 11 00:59:17.651821 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:17.651789    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:17.718559 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:17.718529    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:18.535298 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.535265    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:18.755938 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.755889    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:18.785526 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.785419    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-5nbw5"]
Sep 11 00:59:18.788841 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.788823    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-5nbw5"
Sep 11 00:59:18.791540 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.791251    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 11 00:59:18.791540 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.791363    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 11 00:59:18.791540 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.791462    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 11 00:59:18.791540 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.791478    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 11 00:59:18.792171 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.792154    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 11 00:59:18.799447 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.793263    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-b6p4n\""
Sep 11 00:59:18.803255 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.803233    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-5nbw5"]
Sep 11 00:59:18.836489 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.836465    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9rdkv"]
Sep 11 00:59:18.839943 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.839903    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9rdkv"
Sep 11 00:59:18.843668 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.843652    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 11 00:59:18.845297 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.845282    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vwq6p\""
Sep 11 00:59:18.845731 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.845704    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 11 00:59:18.847007 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.846988    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 11 00:59:18.863928 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.861801    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmjc4\" (UniqueName: \"kubernetes.io/projected/0aa988f8-e121-432c-8be1-90595aeb64a0-kube-api-access-hmjc4\") pod \"openshift-state-metrics-67dcf8c979-5nbw5\" (UID: \"0aa988f8-e121-432c-8be1-90595aeb64a0\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-5nbw5"
Sep 11 00:59:18.863928 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.861869    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0aa988f8-e121-432c-8be1-90595aeb64a0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-5nbw5\" (UID: \"0aa988f8-e121-432c-8be1-90595aeb64a0\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-5nbw5"
Sep 11 00:59:18.863928 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.861899    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0aa988f8-e121-432c-8be1-90595aeb64a0-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-5nbw5\" (UID: \"0aa988f8-e121-432c-8be1-90595aeb64a0\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-5nbw5"
Sep 11 00:59:18.863928 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.861955    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0aa988f8-e121-432c-8be1-90595aeb64a0-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-5nbw5\" (UID: \"0aa988f8-e121-432c-8be1-90595aeb64a0\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-5nbw5"
Sep 11 00:59:18.874971 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.874940    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:18.958462 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.958441    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:18.962967 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.962943    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9b1169c4-a910-4cb0-85f4-dcccb089a5cb-root\") pod \"node-exporter-9rdkv\" (UID: \"9b1169c4-a910-4cb0-85f4-dcccb089a5cb\") " pod="openshift-monitoring/node-exporter-9rdkv"
Sep 11 00:59:18.963059 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.962979    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9b1169c4-a910-4cb0-85f4-dcccb089a5cb-node-exporter-wtmp\") pod \"node-exporter-9rdkv\" (UID: \"9b1169c4-a910-4cb0-85f4-dcccb089a5cb\") " pod="openshift-monitoring/node-exporter-9rdkv"
Sep 11 00:59:18.963059 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.963007    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b1169c4-a910-4cb0-85f4-dcccb089a5cb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9rdkv\" (UID: \"9b1169c4-a910-4cb0-85f4-dcccb089a5cb\") " pod="openshift-monitoring/node-exporter-9rdkv"
Sep 11 00:59:18.963147 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.963081    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9b1169c4-a910-4cb0-85f4-dcccb089a5cb-node-exporter-textfile\") pod \"node-exporter-9rdkv\" (UID: \"9b1169c4-a910-4cb0-85f4-dcccb089a5cb\") " pod="openshift-monitoring/node-exporter-9rdkv"
Sep 11 00:59:18.963147 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.963107    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9b1169c4-a910-4cb0-85f4-dcccb089a5cb-node-exporter-accelerators-collector-config\") pod \"node-exporter-9rdkv\" (UID: \"9b1169c4-a910-4cb0-85f4-dcccb089a5cb\") " pod="openshift-monitoring/node-exporter-9rdkv"
Sep 11 00:59:18.963147 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.963138    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0aa988f8-e121-432c-8be1-90595aeb64a0-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-5nbw5\" (UID: \"0aa988f8-e121-432c-8be1-90595aeb64a0\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-5nbw5"
Sep 11 00:59:18.963280 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.963172    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0aa988f8-e121-432c-8be1-90595aeb64a0-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-5nbw5\" (UID: \"0aa988f8-e121-432c-8be1-90595aeb64a0\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-5nbw5"
Sep 11 00:59:18.963280 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.963194    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b1169c4-a910-4cb0-85f4-dcccb089a5cb-metrics-client-ca\") pod \"node-exporter-9rdkv\" (UID: \"9b1169c4-a910-4cb0-85f4-dcccb089a5cb\") " pod="openshift-monitoring/node-exporter-9rdkv"
Sep 11 00:59:18.963280 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.963224    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9b1169c4-a910-4cb0-85f4-dcccb089a5cb-node-exporter-tls\") pod \"node-exporter-9rdkv\" (UID: \"9b1169c4-a910-4cb0-85f4-dcccb089a5cb\") " pod="openshift-monitoring/node-exporter-9rdkv"
Sep 11 00:59:18.963280 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.963253    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmjc4\" (UniqueName: \"kubernetes.io/projected/0aa988f8-e121-432c-8be1-90595aeb64a0-kube-api-access-hmjc4\") pod \"openshift-state-metrics-67dcf8c979-5nbw5\" (UID: \"0aa988f8-e121-432c-8be1-90595aeb64a0\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-5nbw5"
Sep 11 00:59:18.963463 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.963282    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9b1169c4-a910-4cb0-85f4-dcccb089a5cb-sys\") pod \"node-exporter-9rdkv\" (UID: \"9b1169c4-a910-4cb0-85f4-dcccb089a5cb\") " pod="openshift-monitoring/node-exporter-9rdkv"
Sep 11 00:59:18.963463 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.963329    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0aa988f8-e121-432c-8be1-90595aeb64a0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-5nbw5\" (UID: \"0aa988f8-e121-432c-8be1-90595aeb64a0\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-5nbw5"
Sep 11 00:59:18.963463 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.963390    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqnn7\" (UniqueName: \"kubernetes.io/projected/9b1169c4-a910-4cb0-85f4-dcccb089a5cb-kube-api-access-vqnn7\") pod \"node-exporter-9rdkv\" (UID: \"9b1169c4-a910-4cb0-85f4-dcccb089a5cb\") " pod="openshift-monitoring/node-exporter-9rdkv"
Sep 11 00:59:18.963610 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:59:18.963535    2574 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 11 00:59:18.963610 ip-10-0-139-235 kubenswrapper[2574]: E0911 00:59:18.963604    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0aa988f8-e121-432c-8be1-90595aeb64a0-openshift-state-metrics-tls podName:0aa988f8-e121-432c-8be1-90595aeb64a0 nodeName:}" failed. No retries permitted until 2026-09-11 00:59:19.463570887 +0000 UTC m=+74.860763043 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/0aa988f8-e121-432c-8be1-90595aeb64a0-openshift-state-metrics-tls") pod "openshift-state-metrics-67dcf8c979-5nbw5" (UID: "0aa988f8-e121-432c-8be1-90595aeb64a0") : secret "openshift-state-metrics-tls" not found
Sep 11 00:59:18.963863 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.963841    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0aa988f8-e121-432c-8be1-90595aeb64a0-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-5nbw5\" (UID: \"0aa988f8-e121-432c-8be1-90595aeb64a0\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-5nbw5"
Sep 11 00:59:18.966141 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.966116    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0aa988f8-e121-432c-8be1-90595aeb64a0-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-5nbw5\" (UID: \"0aa988f8-e121-432c-8be1-90595aeb64a0\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-5nbw5"
Sep 11 00:59:18.974955 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:18.974931    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmjc4\" (UniqueName: \"kubernetes.io/projected/0aa988f8-e121-432c-8be1-90595aeb64a0-kube-api-access-hmjc4\") pod \"openshift-state-metrics-67dcf8c979-5nbw5\" (UID: \"0aa988f8-e121-432c-8be1-90595aeb64a0\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-5nbw5"
Sep 11 00:59:19.063999 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:19.063932    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqnn7\" (UniqueName: \"kubernetes.io/projected/9b1169c4-a910-4cb0-85f4-dcccb089a5cb-kube-api-access-vqnn7\") pod \"node-exporter-9rdkv\" (UID: \"9b1169c4-a910-4cb0-85f4-dcccb089a5cb\") " pod="openshift-monitoring/node-exporter-9rdkv"
Sep 11 00:59:19.063999 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:19.063978    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9b1169c4-a910-4cb0-85f4-dcccb089a5cb-root\") pod \"node-exporter-9rdkv\" (UID: \"9b1169c4-a910-4cb0-85f4-dcccb089a5cb\") " pod="openshift-monitoring/node-exporter-9rdkv"
Sep 11 00:59:19.064160 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:19.064004    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9b1169c4-a910-4cb0-85f4-dcccb089a5cb-node-exporter-wtmp\") pod \"node-exporter-9rdkv\" (UID: \"9b1169c4-a910-4cb0-85f4-dcccb089a5cb\") " pod="openshift-monitoring/node-exporter-9rdkv"
Sep 11 00:59:19.064160 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:19.064032    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b1169c4-a910-4cb0-85f4-dcccb089a5cb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9rdkv\" (UID: \"9b1169c4-a910-4cb0-85f4-dcccb089a5cb\") " pod="openshift-monitoring/node-exporter-9rdkv"
Sep 11 00:59:19.064160 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:19.064061    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9b1169c4-a910-4cb0-85f4-dcccb089a5cb-node-exporter-textfile\") pod \"node-exporter-9rdkv\" (UID: \"9b1169c4-a910-4cb0-85f4-dcccb089a5cb\") " pod="openshift-monitoring/node-exporter-9rdkv"
Sep 11 00:59:19.064160 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:19.064118    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9b1169c4-a910-4cb0-85f4-dcccb089a5cb-root\") pod \"node-exporter-9rdkv\" (UID: \"9b1169c4-a910-4cb0-85f4-dcccb089a5cb\") " pod="openshift-monitoring/node-exporter-9rdkv"
Sep 11 00:59:19.064360 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:19.064177    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9b1169c4-a910-4cb0-85f4-dcccb089a5cb-node-exporter-wtmp\") pod \"node-exporter-9rdkv\" (UID: \"9b1169c4-a910-4cb0-85f4-dcccb089a5cb\") " pod="openshift-monitoring/node-exporter-9rdkv"
Sep 11 00:59:19.064464 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:19.064447    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9b1169c4-a910-4cb0-85f4-dcccb089a5cb-node-exporter-textfile\") pod \"node-exporter-9rdkv\" (UID: \"9b1169c4-a910-4cb0-85f4-dcccb089a5cb\") " pod="openshift-monitoring/node-exporter-9rdkv"
Sep 11 00:59:19.064517 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:19.064471    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9b1169c4-a910-4cb0-85f4-dcccb089a5cb-node-exporter-accelerators-collector-config\") pod \"node-exporter-9rdkv\" (UID: \"9b1169c4-a910-4cb0-85f4-dcccb089a5cb\") " pod="openshift-monitoring/node-exporter-9rdkv"
Sep 11 00:59:19.064564 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:19.064515    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b1169c4-a910-4cb0-85f4-dcccb089a5cb-metrics-client-ca\") pod \"node-exporter-9rdkv\" (UID: \"9b1169c4-a910-4cb0-85f4-dcccb089a5cb\") " pod="openshift-monitoring/node-exporter-9rdkv"
Sep 11 00:59:19.064564 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:19.064548    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9b1169c4-a910-4cb0-85f4-dcccb089a5cb-node-exporter-tls\") pod \"node-exporter-9rdkv\" (UID: \"9b1169c4-a910-4cb0-85f4-dcccb089a5cb\") " pod="openshift-monitoring/node-exporter-9rdkv"
Sep 11 00:59:19.064661 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:19.064585    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9b1169c4-a910-4cb0-85f4-dcccb089a5cb-sys\") pod \"node-exporter-9rdkv\" (UID: \"9b1169c4-a910-4cb0-85f4-dcccb089a5cb\") " pod="openshift-monitoring/node-exporter-9rdkv"
Sep 11 00:59:19.064661 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:19.064655    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9b1169c4-a910-4cb0-85f4-dcccb089a5cb-sys\") pod \"node-exporter-9rdkv\" (UID: \"9b1169c4-a910-4cb0-85f4-dcccb089a5cb\") " pod="openshift-monitoring/node-exporter-9rdkv"
Sep 11 00:59:19.065105 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:19.065047    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9b1169c4-a910-4cb0-85f4-dcccb089a5cb-node-exporter-accelerators-collector-config\") pod \"node-exporter-9rdkv\" (UID: \"9b1169c4-a910-4cb0-85f4-dcccb089a5cb\") " pod="openshift-monitoring/node-exporter-9rdkv"
Sep 11 00:59:19.065218 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:19.065184    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b1169c4-a910-4cb0-85f4-dcccb089a5cb-metrics-client-ca\") pod \"node-exporter-9rdkv\" (UID: \"9b1169c4-a910-4cb0-85f4-dcccb089a5cb\") " pod="openshift-monitoring/node-exporter-9rdkv"
Sep 11 00:59:19.066691 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:19.066665    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b1169c4-a910-4cb0-85f4-dcccb089a5cb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9rdkv\" (UID: \"9b1169c4-a910-4cb0-85f4-dcccb089a5cb\") " pod="openshift-monitoring/node-exporter-9rdkv"
Sep 11 00:59:19.067250 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:19.067229    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9b1169c4-a910-4cb0-85f4-dcccb089a5cb-node-exporter-tls\") pod \"node-exporter-9rdkv\" (UID: \"9b1169c4-a910-4cb0-85f4-dcccb089a5cb\") " pod="openshift-monitoring/node-exporter-9rdkv"
Sep 11 00:59:19.071986 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:19.071967    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqnn7\" (UniqueName: \"kubernetes.io/projected/9b1169c4-a910-4cb0-85f4-dcccb089a5cb-kube-api-access-vqnn7\") pod \"node-exporter-9rdkv\" (UID: \"9b1169c4-a910-4cb0-85f4-dcccb089a5cb\") " pod="openshift-monitoring/node-exporter-9rdkv"
Sep 11 00:59:19.157924 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:19.157891    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9rdkv"
Sep 11 00:59:19.170873 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:19.170833    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:19.268338 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:19.268310    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:19.394953 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:19.394875    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:19.467724 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:19.467699    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0aa988f8-e121-432c-8be1-90595aeb64a0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-5nbw5\" (UID: \"0aa988f8-e121-432c-8be1-90595aeb64a0\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-5nbw5"
Sep 11 00:59:19.470033 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:19.470015    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0aa988f8-e121-432c-8be1-90595aeb64a0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-5nbw5\" (UID: \"0aa988f8-e121-432c-8be1-90595aeb64a0\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-5nbw5"
Sep 11 00:59:19.606226 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:19.606193    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9rdkv" event={"ID":"9b1169c4-a910-4cb0-85f4-dcccb089a5cb","Type":"ContainerStarted","Data":"3fb11d70d60399859c6a50ad1dc71a10cd2b5d719a7225de8f69a2cbe76d8c19"}
Sep 11 00:59:19.610093 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:19.610066    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:19.705812 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:19.705786    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-5nbw5"
Sep 11 00:59:19.821082 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:19.821054    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:19.919455 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:19.919421    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:20.001751 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:20.001686    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-5nbw5"]
Sep 11 00:59:20.006785 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:20.006761    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:20.017509 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:59:20.017483    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aa988f8_e121_432c_8be1_90595aeb64a0.slice/crio-59461ea5b4585d06fe62cc9cc960d451b206756114a5a45aa462a5780b8fe1cc WatchSource:0}: Error finding container 59461ea5b4585d06fe62cc9cc960d451b206756114a5a45aa462a5780b8fe1cc: Status 404 returned error can't find the container with id 59461ea5b4585d06fe62cc9cc960d451b206756114a5a45aa462a5780b8fe1cc
Sep 11 00:59:20.221751 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:20.221717    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:20.611801 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:20.611761    2574 generic.go:356] "Generic (PLEG): container finished" podID="9b1169c4-a910-4cb0-85f4-dcccb089a5cb" containerID="7a365b8d8c2b32d80c28dc9b159195fb817d0f86de4454dc89df90f7d31c09a1" exitCode=0
Sep 11 00:59:20.612259 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:20.611850    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9rdkv" event={"ID":"9b1169c4-a910-4cb0-85f4-dcccb089a5cb","Type":"ContainerDied","Data":"7a365b8d8c2b32d80c28dc9b159195fb817d0f86de4454dc89df90f7d31c09a1"}
Sep 11 00:59:20.614081 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:20.614044    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-5nbw5" event={"ID":"0aa988f8-e121-432c-8be1-90595aeb64a0","Type":"ContainerStarted","Data":"f2f7d4474c1a819794d39a009278f49e5a8f0d7199818e1c715975139604bcbe"}
Sep 11 00:59:20.614170 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:20.614102    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-5nbw5" event={"ID":"0aa988f8-e121-432c-8be1-90595aeb64a0","Type":"ContainerStarted","Data":"23cc0eec10803fc4d8c5c572d424c52ca1336fabb27d057ef8d60345724a51ab"}
Sep 11 00:59:20.614170 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:20.614132    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-5nbw5" event={"ID":"0aa988f8-e121-432c-8be1-90595aeb64a0","Type":"ContainerStarted","Data":"59461ea5b4585d06fe62cc9cc960d451b206756114a5a45aa462a5780b8fe1cc"}
Sep 11 00:59:20.708638 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:20.708616    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:20.768478 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:20.768443    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:21.620069 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:21.620038    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9rdkv" event={"ID":"9b1169c4-a910-4cb0-85f4-dcccb089a5cb","Type":"ContainerStarted","Data":"3e801ab979105528320cf33a2a44c782ec6b02d5c72747b8c090aebce831461d"}
Sep 11 00:59:21.620450 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:21.620098    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9rdkv" event={"ID":"9b1169c4-a910-4cb0-85f4-dcccb089a5cb","Type":"ContainerStarted","Data":"4dbe8e049d6d39e5c8aedeb047d61cfac4e40523e1672ecbc849d2fc91f9715c"}
Sep 11 00:59:21.622150 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:21.622128    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-5nbw5" event={"ID":"0aa988f8-e121-432c-8be1-90595aeb64a0","Type":"ContainerStarted","Data":"fe97d46382103ef9db51ca27d4706af96e59888642c494adb5209cd725c56b05"}
Sep 11 00:59:21.639544 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:21.639496    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9rdkv" podStartSLOduration=2.678090916 podStartE2EDuration="3.639483677s" totalImagesPullingTime="889.798224ms" totalInitContainerRuntime="71.594537ms" isStatefulPod=true podCreationTimestamp="2026-09-11 00:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:59:21.638257961 +0000 UTC m=+77.035450146" watchObservedRunningTime="2026-09-11 00:59:21.639483677 +0000 UTC m=+77.036675844"
Sep 11 00:59:21.660298 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:21.660240    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-5nbw5" podStartSLOduration=2.7833700930000003 podStartE2EDuration="3.660227612s" totalImagesPullingTime="876.857519ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:59:21.659460675 +0000 UTC m=+77.056652848" watchObservedRunningTime="2026-09-11 00:59:21.660227612 +0000 UTC m=+77.057419782"
Sep 11 00:59:21.748681 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:21.748654    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:21.845865 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:21.845842    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:21.943865 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:21.943830    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:22.044383 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:22.044355    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:23.218557 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.218524    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-564878c586-bhl5l"]
Sep 11 00:59:23.221591 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.221575    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-564878c586-bhl5l"
Sep 11 00:59:23.225018 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.224993    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 11 00:59:23.225138 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.225042    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 11 00:59:23.225642 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.225616    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-q8tkn\""
Sep 11 00:59:23.225642 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.225641    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 11 00:59:23.225826 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.225734    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 11 00:59:23.225927 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.225895    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-78j9qph3hr30t\""
Sep 11 00:59:23.232963 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.232942    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-564878c586-bhl5l"]
Sep 11 00:59:23.296198 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.296162    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b808fe3-eb44-4658-bb1e-524852e8fb66-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-564878c586-bhl5l\" (UID: \"2b808fe3-eb44-4658-bb1e-524852e8fb66\") " pod="openshift-monitoring/metrics-server-564878c586-bhl5l"
Sep 11 00:59:23.296359 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.296207    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2b808fe3-eb44-4658-bb1e-524852e8fb66-client-ca-bundle\") pod \"metrics-server-564878c586-bhl5l\" (UID: \"2b808fe3-eb44-4658-bb1e-524852e8fb66\") " pod="openshift-monitoring/metrics-server-564878c586-bhl5l"
Sep 11 00:59:23.296359 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.296237    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8j5s\" (UniqueName: \"kubernetes.io/projected/2b808fe3-eb44-4658-bb1e-524852e8fb66-kube-api-access-h8j5s\") pod \"metrics-server-564878c586-bhl5l\" (UID: \"2b808fe3-eb44-4658-bb1e-524852e8fb66\") " pod="openshift-monitoring/metrics-server-564878c586-bhl5l"
Sep 11 00:59:23.296359 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.296297    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2b808fe3-eb44-4658-bb1e-524852e8fb66-secret-metrics-server-client-certs\") pod \"metrics-server-564878c586-bhl5l\" (UID: \"2b808fe3-eb44-4658-bb1e-524852e8fb66\") " pod="openshift-monitoring/metrics-server-564878c586-bhl5l"
Sep 11 00:59:23.296359 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.296319    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2b808fe3-eb44-4658-bb1e-524852e8fb66-metrics-server-audit-profiles\") pod \"metrics-server-564878c586-bhl5l\" (UID: \"2b808fe3-eb44-4658-bb1e-524852e8fb66\") " pod="openshift-monitoring/metrics-server-564878c586-bhl5l"
Sep 11 00:59:23.296359 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.296349    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2b808fe3-eb44-4658-bb1e-524852e8fb66-secret-metrics-server-tls\") pod \"metrics-server-564878c586-bhl5l\" (UID: \"2b808fe3-eb44-4658-bb1e-524852e8fb66\") " pod="openshift-monitoring/metrics-server-564878c586-bhl5l"
Sep 11 00:59:23.296540 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.296364    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2b808fe3-eb44-4658-bb1e-524852e8fb66-audit-log\") pod \"metrics-server-564878c586-bhl5l\" (UID: \"2b808fe3-eb44-4658-bb1e-524852e8fb66\") " pod="openshift-monitoring/metrics-server-564878c586-bhl5l"
Sep 11 00:59:23.397071 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.397045    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2b808fe3-eb44-4658-bb1e-524852e8fb66-client-ca-bundle\") pod \"metrics-server-564878c586-bhl5l\" (UID: \"2b808fe3-eb44-4658-bb1e-524852e8fb66\") " pod="openshift-monitoring/metrics-server-564878c586-bhl5l"
Sep 11 00:59:23.397190 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.397080    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8j5s\" (UniqueName: \"kubernetes.io/projected/2b808fe3-eb44-4658-bb1e-524852e8fb66-kube-api-access-h8j5s\") pod \"metrics-server-564878c586-bhl5l\" (UID: \"2b808fe3-eb44-4658-bb1e-524852e8fb66\") " pod="openshift-monitoring/metrics-server-564878c586-bhl5l"
Sep 11 00:59:23.397190 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.397111    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2b808fe3-eb44-4658-bb1e-524852e8fb66-secret-metrics-server-client-certs\") pod \"metrics-server-564878c586-bhl5l\" (UID: \"2b808fe3-eb44-4658-bb1e-524852e8fb66\") " pod="openshift-monitoring/metrics-server-564878c586-bhl5l"
Sep 11 00:59:23.397190 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.397136    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2b808fe3-eb44-4658-bb1e-524852e8fb66-metrics-server-audit-profiles\") pod \"metrics-server-564878c586-bhl5l\" (UID: \"2b808fe3-eb44-4658-bb1e-524852e8fb66\") " pod="openshift-monitoring/metrics-server-564878c586-bhl5l"
Sep 11 00:59:23.397190 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.397179    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2b808fe3-eb44-4658-bb1e-524852e8fb66-secret-metrics-server-tls\") pod \"metrics-server-564878c586-bhl5l\" (UID: \"2b808fe3-eb44-4658-bb1e-524852e8fb66\") " pod="openshift-monitoring/metrics-server-564878c586-bhl5l"
Sep 11 00:59:23.397370 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.397201    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2b808fe3-eb44-4658-bb1e-524852e8fb66-audit-log\") pod \"metrics-server-564878c586-bhl5l\" (UID: \"2b808fe3-eb44-4658-bb1e-524852e8fb66\") " pod="openshift-monitoring/metrics-server-564878c586-bhl5l"
Sep 11 00:59:23.397370 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.397230    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b808fe3-eb44-4658-bb1e-524852e8fb66-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-564878c586-bhl5l\" (UID: \"2b808fe3-eb44-4658-bb1e-524852e8fb66\") " pod="openshift-monitoring/metrics-server-564878c586-bhl5l"
Sep 11 00:59:23.397700 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.397675    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2b808fe3-eb44-4658-bb1e-524852e8fb66-audit-log\") pod \"metrics-server-564878c586-bhl5l\" (UID: \"2b808fe3-eb44-4658-bb1e-524852e8fb66\") " pod="openshift-monitoring/metrics-server-564878c586-bhl5l"
Sep 11 00:59:23.398024 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.398004    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b808fe3-eb44-4658-bb1e-524852e8fb66-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-564878c586-bhl5l\" (UID: \"2b808fe3-eb44-4658-bb1e-524852e8fb66\") " pod="openshift-monitoring/metrics-server-564878c586-bhl5l"
Sep 11 00:59:23.398466 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.398442    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2b808fe3-eb44-4658-bb1e-524852e8fb66-metrics-server-audit-profiles\") pod \"metrics-server-564878c586-bhl5l\" (UID: \"2b808fe3-eb44-4658-bb1e-524852e8fb66\") " pod="openshift-monitoring/metrics-server-564878c586-bhl5l"
Sep 11 00:59:23.399800 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.399781    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2b808fe3-eb44-4658-bb1e-524852e8fb66-client-ca-bundle\") pod \"metrics-server-564878c586-bhl5l\" (UID: \"2b808fe3-eb44-4658-bb1e-524852e8fb66\") " pod="openshift-monitoring/metrics-server-564878c586-bhl5l"
Sep 11 00:59:23.399879 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.399855    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2b808fe3-eb44-4658-bb1e-524852e8fb66-secret-metrics-server-client-certs\") pod \"metrics-server-564878c586-bhl5l\" (UID: \"2b808fe3-eb44-4658-bb1e-524852e8fb66\") " pod="openshift-monitoring/metrics-server-564878c586-bhl5l"
Sep 11 00:59:23.399950 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.399935    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2b808fe3-eb44-4658-bb1e-524852e8fb66-secret-metrics-server-tls\") pod \"metrics-server-564878c586-bhl5l\" (UID: \"2b808fe3-eb44-4658-bb1e-524852e8fb66\") " pod="openshift-monitoring/metrics-server-564878c586-bhl5l"
Sep 11 00:59:23.406451 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.406432    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8j5s\" (UniqueName: \"kubernetes.io/projected/2b808fe3-eb44-4658-bb1e-524852e8fb66-kube-api-access-h8j5s\") pod \"metrics-server-564878c586-bhl5l\" (UID: \"2b808fe3-eb44-4658-bb1e-524852e8fb66\") " pod="openshift-monitoring/metrics-server-564878c586-bhl5l"
Sep 11 00:59:23.530807 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.530709    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-564878c586-bhl5l"
Sep 11 00:59:23.591686 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.591656    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:23.671252 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.671219    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:23.805386 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.805311    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-564878c586-bhl5l"]
Sep 11 00:59:23.810840 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:59:23.810817    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b808fe3_eb44_4658_bb1e_524852e8fb66.slice/crio-3ce5785c4c4885499ced0dd9aab63cd82eec9045420b5507dca7d43ae098c6b5 WatchSource:0}: Error finding container 3ce5785c4c4885499ced0dd9aab63cd82eec9045420b5507dca7d43ae098c6b5: Status 404 returned error can't find the container with id 3ce5785c4c4885499ced0dd9aab63cd82eec9045420b5507dca7d43ae098c6b5
Sep 11 00:59:23.899584 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:23.899559    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:24.633049 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:24.633017    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-564878c586-bhl5l" event={"ID":"2b808fe3-eb44-4658-bb1e-524852e8fb66","Type":"ContainerStarted","Data":"3ce5785c4c4885499ced0dd9aab63cd82eec9045420b5507dca7d43ae098c6b5"}
Sep 11 00:59:25.041335 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.041307    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:25.047106 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.047083    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 00:59:25.053355 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.053338    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.056131 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.056112    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 11 00:59:25.056222 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.056116    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-h8mht\""
Sep 11 00:59:25.056222 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.056211    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 11 00:59:25.056335 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.056220    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 11 00:59:25.056437 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.056424    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 11 00:59:25.056491 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.056448    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 11 00:59:25.056549 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.056454    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 11 00:59:25.056745 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.056722    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 11 00:59:25.058370 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.058354    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 11 00:59:25.058370 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.058365    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 11 00:59:25.058538 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.058354    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 11 00:59:25.059396 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.059375    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 11 00:59:25.059999 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.059903    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 11 00:59:25.059999 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.059982    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-mai91c4a443g\""
Sep 11 00:59:25.060128 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.059986    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 11 00:59:25.061878 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.061844    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 11 00:59:25.074248 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.074222    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 00:59:25.111480 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.111453    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-web-config\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.111572 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.111518    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.111572 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.111548    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.111662 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.111582    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.111662 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.111609    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.111662 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.111634    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.111662 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.111654    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.111804 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.111698    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.111804 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.111742    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7f8f5157-da76-4270-973d-bebc2d3b9e74-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.111804 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.111765    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.111804 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.111780    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.111804 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.111801    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.111984 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.111822    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.111984 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.111878    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.111984 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.111938    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7f8f5157-da76-4270-973d-bebc2d3b9e74-config-out\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.111984 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.111965    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dk77\" (UniqueName: \"kubernetes.io/projected/7f8f5157-da76-4270-973d-bebc2d3b9e74-kube-api-access-9dk77\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.112118 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.112000    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.112118 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.112023    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7f8f5157-da76-4270-973d-bebc2d3b9e74-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.112118 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.112053    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-config\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.112118 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.112078    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.213030 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.213002    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.213030 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.213032    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7f8f5157-da76-4270-973d-bebc2d3b9e74-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.213177 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.213051    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.213177 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.213066    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.213285 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.213267    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.213337 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.213309    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.213390 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.213341    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.213390 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.213370    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7f8f5157-da76-4270-973d-bebc2d3b9e74-config-out\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.213493 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.213395    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dk77\" (UniqueName: \"kubernetes.io/projected/7f8f5157-da76-4270-973d-bebc2d3b9e74-kube-api-access-9dk77\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.213493 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.213428    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.213493 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.213451    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7f8f5157-da76-4270-973d-bebc2d3b9e74-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.213493 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.213485    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-config\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.213680 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.213508    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.213680 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.213551    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-web-config\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.213680 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.213609    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.213680 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.213635    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.213680 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.213673    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.213940 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.213704    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.213940 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.213729    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.213940 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.213754    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.214106 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.214017    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7f8f5157-da76-4270-973d-bebc2d3b9e74-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.216272 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.216246    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.216501 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.216472    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.216940 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.216728    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.216940 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.213727    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.216940 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.216793    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.217146 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.217057    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.217615 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.217273    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.217615 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.217302    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.217615 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.217350    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-config\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.217809 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.217679    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.217868 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.217832    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7f8f5157-da76-4270-973d-bebc2d3b9e74-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.218568 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.218521    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.218663 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.218581    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.219240 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.219216    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.219335 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.219312    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-web-config\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.219840 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.219819    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.220047 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.220028    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7f8f5157-da76-4270-973d-bebc2d3b9e74-config-out\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.220736 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.220717    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.227342 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.227321    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dk77\" (UniqueName: \"kubernetes.io/projected/7f8f5157-da76-4270-973d-bebc2d3b9e74-kube-api-access-9dk77\") pod \"prometheus-k8s-0\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.261731 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.261705    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:25.364533 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.364449    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:25.383017 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.382994    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:25.638290 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.638213    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-564878c586-bhl5l" event={"ID":"2b808fe3-eb44-4658-bb1e-524852e8fb66","Type":"ContainerStarted","Data":"043dd0287cebf2503e93f54bf4b587b334ee01cf416bbdcda214e067043d3708"}
Sep 11 00:59:25.661114 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.660987    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-564878c586-bhl5l" podStartSLOduration=1.448393361 podStartE2EDuration="2.660968719s" totalImagesPullingTime="1.212575358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:59:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:59:25.660386738 +0000 UTC m=+81.057578910" watchObservedRunningTime="2026-09-11 00:59:25.660968719 +0000 UTC m=+81.058160893"
Sep 11 00:59:25.679109 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.679069    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 00:59:25.681627 ip-10-0-139-235 kubenswrapper[2574]: W0911 00:59:25.681604    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f8f5157_da76_4270_973d_bebc2d3b9e74.slice/crio-6701bd0987a0c327dc189ad60c095fc8775fd460d1eb3c13a07e205030da3a17 WatchSource:0}: Error finding container 6701bd0987a0c327dc189ad60c095fc8775fd460d1eb3c13a07e205030da3a17: Status 404 returned error can't find the container with id 6701bd0987a0c327dc189ad60c095fc8775fd460d1eb3c13a07e205030da3a17
Sep 11 00:59:25.970157 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:25.970130    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:26.071045 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:26.071015    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:26.643285 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:26.643246    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7f8f5157-da76-4270-973d-bebc2d3b9e74","Type":"ContainerStarted","Data":"6701bd0987a0c327dc189ad60c095fc8775fd460d1eb3c13a07e205030da3a17"}
Sep 11 00:59:26.707314 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:26.707293    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:26.768857 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:26.768833    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:26.883960 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:26.883882    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:26.945393 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:26.945365    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:27.549677 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:27.549653    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-84ccd59f5-rz4qk"
Sep 11 00:59:27.647947 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:27.647895    2574 generic.go:356] "Generic (PLEG): container finished" podID="7f8f5157-da76-4270-973d-bebc2d3b9e74" containerID="0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79" exitCode=0
Sep 11 00:59:27.647947 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:27.647939    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7f8f5157-da76-4270-973d-bebc2d3b9e74","Type":"ContainerDied","Data":"0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79"}
Sep 11 00:59:27.744325 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:27.744290    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:27.843048 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:27.842979    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:28.152830 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:28.152727    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:28.254278 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:28.254248    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:28.969633 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:28.969541    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:31.666361 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:31.666321    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7f8f5157-da76-4270-973d-bebc2d3b9e74","Type":"ContainerStarted","Data":"aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3"}
Sep 11 00:59:31.666844 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:31.666427    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7f8f5157-da76-4270-973d-bebc2d3b9e74","Type":"ContainerStarted","Data":"c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768"}
Sep 11 00:59:32.673618 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:32.673540    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7f8f5157-da76-4270-973d-bebc2d3b9e74","Type":"ContainerStarted","Data":"84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610"}
Sep 11 00:59:32.673618 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:32.673604    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7f8f5157-da76-4270-973d-bebc2d3b9e74","Type":"ContainerStarted","Data":"736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5"}
Sep 11 00:59:32.673618 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:32.673629    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7f8f5157-da76-4270-973d-bebc2d3b9e74","Type":"ContainerStarted","Data":"50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7"}
Sep 11 00:59:33.681376 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:33.681343    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7f8f5157-da76-4270-973d-bebc2d3b9e74","Type":"ContainerStarted","Data":"55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b"}
Sep 11 00:59:33.708567 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:33.708518    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.100546018 podStartE2EDuration="8.708500841s" totalImagesPullingTime="5.590665414s" totalInitContainerRuntime="17.289409ms" isStatefulPod=false podCreationTimestamp="2026-09-11 00:59:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:59:33.705959757 +0000 UTC m=+89.103151935" watchObservedRunningTime="2026-09-11 00:59:33.708500841 +0000 UTC m=+89.105693014"
Sep 11 00:59:34.404271 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:34.404237    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:34.505100 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:34.505070    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:35.384153 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:35.384121    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 00:59:35.692622 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:35.692592    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:35.783962 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:35.783930    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:35.854611 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:35.854559    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:35.966562 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:35.966482    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:36.066826 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:36.066798    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:43.531434 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:43.531399    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-564878c586-bhl5l"
Sep 11 00:59:43.531860 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:43.531469    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-564878c586-bhl5l"
Sep 11 00:59:46.594640 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:46.594614    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9hl4x"
Sep 11 00:59:50.700291 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:50.700254    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 00:59:50.757172 ip-10-0-139-235 kubenswrapper[2574]: I0911 00:59:50.757134    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:00:02.060500 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:02.060456    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:00:02.309219 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:02.309183    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:00:03.536403 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:03.536370    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-564878c586-bhl5l"
Sep 11 01:00:03.540242 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:03.540212    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-564878c586-bhl5l"
Sep 11 01:00:24.358245 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:24.358192    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:00:25.384110 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:25.384083    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:25.404162 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:25.404134    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:25.880770 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:25.880745    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:31.843795 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:31.843761    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:00:38.075711 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:38.075675    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:00:38.973817 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:38.973782    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:00:43.463159 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.463127    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 01:00:43.463482 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.463410    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7f8f5157-da76-4270-973d-bebc2d3b9e74" containerName="prometheus" containerID="cri-o://c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768" gracePeriod=600
Sep 11 01:00:43.463482 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.463438    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7f8f5157-da76-4270-973d-bebc2d3b9e74" containerName="kube-rbac-proxy" containerID="cri-o://84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610" gracePeriod=600
Sep 11 01:00:43.463602 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.463479    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7f8f5157-da76-4270-973d-bebc2d3b9e74" containerName="thanos-sidecar" containerID="cri-o://50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7" gracePeriod=600
Sep 11 01:00:43.463602 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.463485    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7f8f5157-da76-4270-973d-bebc2d3b9e74" containerName="kube-rbac-proxy-thanos" containerID="cri-o://55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b" gracePeriod=600
Sep 11 01:00:43.463602 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.463522    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7f8f5157-da76-4270-973d-bebc2d3b9e74" containerName="config-reloader" containerID="cri-o://aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3" gracePeriod=600
Sep 11 01:00:43.463602 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.463493    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7f8f5157-da76-4270-973d-bebc2d3b9e74" containerName="kube-rbac-proxy-web" containerID="cri-o://736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5" gracePeriod=600
Sep 11 01:00:43.699611 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.699583    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:43.831835 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.831755    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-prometheus-k8s-rulefiles-1\") pod \"7f8f5157-da76-4270-973d-bebc2d3b9e74\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") "
Sep 11 01:00:43.831835 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.831795    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-prometheus-trusted-ca-bundle\") pod \"7f8f5157-da76-4270-973d-bebc2d3b9e74\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") "
Sep 11 01:00:43.831835 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.831818    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"7f8f5157-da76-4270-973d-bebc2d3b9e74\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") "
Sep 11 01:00:43.832134 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.831865    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-grpc-tls\") pod \"7f8f5157-da76-4270-973d-bebc2d3b9e74\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") "
Sep 11 01:00:43.832134 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.831895    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9dk77\" (UniqueName: \"kubernetes.io/projected/7f8f5157-da76-4270-973d-bebc2d3b9e74-kube-api-access-9dk77\") pod \"7f8f5157-da76-4270-973d-bebc2d3b9e74\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") "
Sep 11 01:00:43.832134 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.831958    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-configmap-metrics-client-ca\") pod \"7f8f5157-da76-4270-973d-bebc2d3b9e74\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") "
Sep 11 01:00:43.832134 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.831998    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"7f8f5157-da76-4270-973d-bebc2d3b9e74\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") "
Sep 11 01:00:43.832134 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.832022    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-kube-rbac-proxy\") pod \"7f8f5157-da76-4270-973d-bebc2d3b9e74\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") "
Sep 11 01:00:43.832134 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.832050    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-configmap-kubelet-serving-ca-bundle\") pod \"7f8f5157-da76-4270-973d-bebc2d3b9e74\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") "
Sep 11 01:00:43.832429 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.832197    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "7f8f5157-da76-4270-973d-bebc2d3b9e74" (UID: "7f8f5157-da76-4270-973d-bebc2d3b9e74"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:00:43.832429 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.832268    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "7f8f5157-da76-4270-973d-bebc2d3b9e74" (UID: "7f8f5157-da76-4270-973d-bebc2d3b9e74"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:00:43.832429 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.832328    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-metrics-client-certs\") pod \"7f8f5157-da76-4270-973d-bebc2d3b9e74\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") "
Sep 11 01:00:43.832429 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.832361    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7f8f5157-da76-4270-973d-bebc2d3b9e74-prometheus-k8s-db\") pod \"7f8f5157-da76-4270-973d-bebc2d3b9e74\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") "
Sep 11 01:00:43.832622 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.832493    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-thanos-prometheus-http-client-file\") pod \"7f8f5157-da76-4270-973d-bebc2d3b9e74\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") "
Sep 11 01:00:43.832622 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.832524    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-config\") pod \"7f8f5157-da76-4270-973d-bebc2d3b9e74\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") "
Sep 11 01:00:43.832622 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.832571    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-web-config\") pod \"7f8f5157-da76-4270-973d-bebc2d3b9e74\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") "
Sep 11 01:00:43.832622 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.832596    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7f8f5157-da76-4270-973d-bebc2d3b9e74-config-out\") pod \"7f8f5157-da76-4270-973d-bebc2d3b9e74\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") "
Sep 11 01:00:43.832827 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.832625    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7f8f5157-da76-4270-973d-bebc2d3b9e74-tls-assets\") pod \"7f8f5157-da76-4270-973d-bebc2d3b9e74\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") "
Sep 11 01:00:43.832827 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.832650    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-prometheus-k8s-rulefiles-0\") pod \"7f8f5157-da76-4270-973d-bebc2d3b9e74\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") "
Sep 11 01:00:43.832827 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.832681    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-prometheus-k8s-tls\") pod \"7f8f5157-da76-4270-973d-bebc2d3b9e74\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") "
Sep 11 01:00:43.833002 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.832966    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-configmap-serving-certs-ca-bundle\") pod \"7f8f5157-da76-4270-973d-bebc2d3b9e74\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") "
Sep 11 01:00:43.833002 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.832980    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "7f8f5157-da76-4270-973d-bebc2d3b9e74" (UID: "7f8f5157-da76-4270-973d-bebc2d3b9e74"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:00:43.833106 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.833005    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-prometheus-k8s-rulefiles-2\") pod \"7f8f5157-da76-4270-973d-bebc2d3b9e74\" (UID: \"7f8f5157-da76-4270-973d-bebc2d3b9e74\") "
Sep 11 01:00:43.833106 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.833012    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "7f8f5157-da76-4270-973d-bebc2d3b9e74" (UID: "7f8f5157-da76-4270-973d-bebc2d3b9e74"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:00:43.833248 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.833232    2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:00:43.833312 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.833254    2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-prometheus-trusted-ca-bundle\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:00:43.833312 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.833300    2574 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-configmap-metrics-client-ca\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:00:43.833411 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.833316    2574 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:00:43.833662 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.833641    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "7f8f5157-da76-4270-973d-bebc2d3b9e74" (UID: "7f8f5157-da76-4270-973d-bebc2d3b9e74"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:00:43.835296 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.834698    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "7f8f5157-da76-4270-973d-bebc2d3b9e74" (UID: "7f8f5157-da76-4270-973d-bebc2d3b9e74"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:00:43.835296 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.835245    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f8f5157-da76-4270-973d-bebc2d3b9e74-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "7f8f5157-da76-4270-973d-bebc2d3b9e74" (UID: "7f8f5157-da76-4270-973d-bebc2d3b9e74"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:00:43.835452 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.835434    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "7f8f5157-da76-4270-973d-bebc2d3b9e74" (UID: "7f8f5157-da76-4270-973d-bebc2d3b9e74"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:00:43.835942 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.835890    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "7f8f5157-da76-4270-973d-bebc2d3b9e74" (UID: "7f8f5157-da76-4270-973d-bebc2d3b9e74"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:00:43.836033 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.835929    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "7f8f5157-da76-4270-973d-bebc2d3b9e74" (UID: "7f8f5157-da76-4270-973d-bebc2d3b9e74"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:00:43.836033 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.836002    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f8f5157-da76-4270-973d-bebc2d3b9e74-kube-api-access-9dk77" (OuterVolumeSpecName: "kube-api-access-9dk77") pod "7f8f5157-da76-4270-973d-bebc2d3b9e74" (UID: "7f8f5157-da76-4270-973d-bebc2d3b9e74"). InnerVolumeSpecName "kube-api-access-9dk77". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:00:43.836264 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.836236    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "7f8f5157-da76-4270-973d-bebc2d3b9e74" (UID: "7f8f5157-da76-4270-973d-bebc2d3b9e74"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:00:43.836673 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.836644    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "7f8f5157-da76-4270-973d-bebc2d3b9e74" (UID: "7f8f5157-da76-4270-973d-bebc2d3b9e74"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:00:43.836799 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.836765    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "7f8f5157-da76-4270-973d-bebc2d3b9e74" (UID: "7f8f5157-da76-4270-973d-bebc2d3b9e74"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:00:43.837120 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.837093    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "7f8f5157-da76-4270-973d-bebc2d3b9e74" (UID: "7f8f5157-da76-4270-973d-bebc2d3b9e74"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:00:43.837628 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.837603    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f8f5157-da76-4270-973d-bebc2d3b9e74-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "7f8f5157-da76-4270-973d-bebc2d3b9e74" (UID: "7f8f5157-da76-4270-973d-bebc2d3b9e74"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:00:43.837831 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.837810    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f8f5157-da76-4270-973d-bebc2d3b9e74-config-out" (OuterVolumeSpecName: "config-out") pod "7f8f5157-da76-4270-973d-bebc2d3b9e74" (UID: "7f8f5157-da76-4270-973d-bebc2d3b9e74"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:00:43.837956 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.837938    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-config" (OuterVolumeSpecName: "config") pod "7f8f5157-da76-4270-973d-bebc2d3b9e74" (UID: "7f8f5157-da76-4270-973d-bebc2d3b9e74"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:00:43.838108 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.838090    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "7f8f5157-da76-4270-973d-bebc2d3b9e74" (UID: "7f8f5157-da76-4270-973d-bebc2d3b9e74"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:00:43.845786 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.845767    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-web-config" (OuterVolumeSpecName: "web-config") pod "7f8f5157-da76-4270-973d-bebc2d3b9e74" (UID: "7f8f5157-da76-4270-973d-bebc2d3b9e74"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:00:43.928489 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.928458    2574 generic.go:356] "Generic (PLEG): container finished" podID="7f8f5157-da76-4270-973d-bebc2d3b9e74" containerID="55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b" exitCode=0
Sep 11 01:00:43.928489 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.928483    2574 generic.go:356] "Generic (PLEG): container finished" podID="7f8f5157-da76-4270-973d-bebc2d3b9e74" containerID="84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610" exitCode=0
Sep 11 01:00:43.928489 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.928491    2574 generic.go:356] "Generic (PLEG): container finished" podID="7f8f5157-da76-4270-973d-bebc2d3b9e74" containerID="736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5" exitCode=0
Sep 11 01:00:43.928740 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.928497    2574 generic.go:356] "Generic (PLEG): container finished" podID="7f8f5157-da76-4270-973d-bebc2d3b9e74" containerID="50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7" exitCode=0
Sep 11 01:00:43.928740 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.928502    2574 generic.go:356] "Generic (PLEG): container finished" podID="7f8f5157-da76-4270-973d-bebc2d3b9e74" containerID="aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3" exitCode=0
Sep 11 01:00:43.928740 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.928509    2574 generic.go:356] "Generic (PLEG): container finished" podID="7f8f5157-da76-4270-973d-bebc2d3b9e74" containerID="c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768" exitCode=0
Sep 11 01:00:43.928740 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.928532    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:43.928740 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.928533    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7f8f5157-da76-4270-973d-bebc2d3b9e74","Type":"ContainerDied","Data":"55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b"}
Sep 11 01:00:43.928740 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.928621    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7f8f5157-da76-4270-973d-bebc2d3b9e74","Type":"ContainerDied","Data":"84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610"}
Sep 11 01:00:43.928740 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.928665    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7f8f5157-da76-4270-973d-bebc2d3b9e74","Type":"ContainerDied","Data":"736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5"}
Sep 11 01:00:43.928740 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.928675    2574 scope.go:117] "RemoveContainer" containerID="55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b"
Sep 11 01:00:43.928740 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.928710    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7f8f5157-da76-4270-973d-bebc2d3b9e74","Type":"ContainerDied","Data":"50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7"}
Sep 11 01:00:43.929222 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.928750    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7f8f5157-da76-4270-973d-bebc2d3b9e74","Type":"ContainerDied","Data":"aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3"}
Sep 11 01:00:43.929222 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.928791    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7f8f5157-da76-4270-973d-bebc2d3b9e74","Type":"ContainerDied","Data":"c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768"}
Sep 11 01:00:43.929222 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.928835    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7f8f5157-da76-4270-973d-bebc2d3b9e74","Type":"ContainerDied","Data":"6701bd0987a0c327dc189ad60c095fc8775fd460d1eb3c13a07e205030da3a17"}
Sep 11 01:00:43.933776 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.933756    2574 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:00:43.933776 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.933776    2574 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-grpc-tls\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:00:43.933925 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.933787    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dk77\" (UniqueName: \"kubernetes.io/projected/7f8f5157-da76-4270-973d-bebc2d3b9e74-kube-api-access-9dk77\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:00:43.933925 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.933796    2574 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:00:43.933925 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.933807    2574 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-kube-rbac-proxy\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:00:43.933925 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.933819    2574 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-metrics-client-certs\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:00:43.933925 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.933830    2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7f8f5157-da76-4270-973d-bebc2d3b9e74-prometheus-k8s-db\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:00:43.933925 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.933841    2574 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-thanos-prometheus-http-client-file\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:00:43.933925 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.933850    2574 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-config\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:00:43.933925 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.933861    2574 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-web-config\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:00:43.933925 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.933870    2574 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7f8f5157-da76-4270-973d-bebc2d3b9e74-config-out\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:00:43.933925 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.933878    2574 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7f8f5157-da76-4270-973d-bebc2d3b9e74-tls-assets\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:00:43.933925 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.933885    2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:00:43.933925 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.933898    2574 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7f8f5157-da76-4270-973d-bebc2d3b9e74-secret-prometheus-k8s-tls\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:00:43.933925 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.933925    2574 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:00:43.933925 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.933934    2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7f8f5157-da76-4270-973d-bebc2d3b9e74-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:00:43.937356 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.937334    2574 scope.go:117] "RemoveContainer" containerID="84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610"
Sep 11 01:00:43.945503 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.945483    2574 scope.go:117] "RemoveContainer" containerID="736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5"
Sep 11 01:00:43.952009 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.951987    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 01:00:43.952375 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.952362    2574 scope.go:117] "RemoveContainer" containerID="50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7"
Sep 11 01:00:43.956136 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.956118    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 01:00:43.959258 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.959241    2574 scope.go:117] "RemoveContainer" containerID="aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3"
Sep 11 01:00:43.965469 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.965451    2574 scope.go:117] "RemoveContainer" containerID="c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768"
Sep 11 01:00:43.972347 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.972332    2574 scope.go:117] "RemoveContainer" containerID="0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79"
Sep 11 01:00:43.978661 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.978644    2574 scope.go:117] "RemoveContainer" containerID="55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b"
Sep 11 01:00:43.978944 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:00:43.978902    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b\": container with ID starting with 55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b not found: ID does not exist" containerID="55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b"
Sep 11 01:00:43.979000 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.978953    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b"} err="failed to get container status \"55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b\": rpc error: code = NotFound desc = could not find container \"55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b\": container with ID starting with 55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b not found: ID does not exist"
Sep 11 01:00:43.979000 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.978984    2574 scope.go:117] "RemoveContainer" containerID="84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610"
Sep 11 01:00:43.979250 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:00:43.979233    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610\": container with ID starting with 84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610 not found: ID does not exist" containerID="84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610"
Sep 11 01:00:43.979315 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.979259    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610"} err="failed to get container status \"84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610\": rpc error: code = NotFound desc = could not find container \"84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610\": container with ID starting with 84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610 not found: ID does not exist"
Sep 11 01:00:43.979315 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.979281    2574 scope.go:117] "RemoveContainer" containerID="736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5"
Sep 11 01:00:43.979550 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:00:43.979533    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5\": container with ID starting with 736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5 not found: ID does not exist" containerID="736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5"
Sep 11 01:00:43.979588 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.979557    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5"} err="failed to get container status \"736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5\": rpc error: code = NotFound desc = could not find container \"736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5\": container with ID starting with 736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5 not found: ID does not exist"
Sep 11 01:00:43.979588 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.979571    2574 scope.go:117] "RemoveContainer" containerID="50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7"
Sep 11 01:00:43.979812 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:00:43.979795    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7\": container with ID starting with 50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7 not found: ID does not exist" containerID="50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7"
Sep 11 01:00:43.979860 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.979816    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7"} err="failed to get container status \"50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7\": rpc error: code = NotFound desc = could not find container \"50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7\": container with ID starting with 50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7 not found: ID does not exist"
Sep 11 01:00:43.979860 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.979829    2574 scope.go:117] "RemoveContainer" containerID="aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3"
Sep 11 01:00:43.980077 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:00:43.980061    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3\": container with ID starting with aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3 not found: ID does not exist" containerID="aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3"
Sep 11 01:00:43.980139 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.980079    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3"} err="failed to get container status \"aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3\": rpc error: code = NotFound desc = could not find container \"aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3\": container with ID starting with aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3 not found: ID does not exist"
Sep 11 01:00:43.980139 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.980092    2574 scope.go:117] "RemoveContainer" containerID="c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768"
Sep 11 01:00:43.980303 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:00:43.980291    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768\": container with ID starting with c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768 not found: ID does not exist" containerID="c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768"
Sep 11 01:00:43.980341 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.980306    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768"} err="failed to get container status \"c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768\": rpc error: code = NotFound desc = could not find container \"c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768\": container with ID starting with c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768 not found: ID does not exist"
Sep 11 01:00:43.980341 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.980317    2574 scope.go:117] "RemoveContainer" containerID="0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79"
Sep 11 01:00:43.980549 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:00:43.980532    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79\": container with ID starting with 0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79 not found: ID does not exist" containerID="0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79"
Sep 11 01:00:43.980591 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.980553    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79"} err="failed to get container status \"0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79\": rpc error: code = NotFound desc = could not find container \"0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79\": container with ID starting with 0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79 not found: ID does not exist"
Sep 11 01:00:43.980591 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.980568    2574 scope.go:117] "RemoveContainer" containerID="55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b"
Sep 11 01:00:43.980795 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.980776    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b"} err="failed to get container status \"55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b\": rpc error: code = NotFound desc = could not find container \"55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b\": container with ID starting with 55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b not found: ID does not exist"
Sep 11 01:00:43.980861 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.980797    2574 scope.go:117] "RemoveContainer" containerID="84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610"
Sep 11 01:00:43.981063 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.981045    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610"} err="failed to get container status \"84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610\": rpc error: code = NotFound desc = could not find container \"84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610\": container with ID starting with 84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610 not found: ID does not exist"
Sep 11 01:00:43.981128 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.981063    2574 scope.go:117] "RemoveContainer" containerID="736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5"
Sep 11 01:00:43.981287 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.981271    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5"} err="failed to get container status \"736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5\": rpc error: code = NotFound desc = could not find container \"736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5\": container with ID starting with 736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5 not found: ID does not exist"
Sep 11 01:00:43.981362 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.981288    2574 scope.go:117] "RemoveContainer" containerID="50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7"
Sep 11 01:00:43.981527 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.981500    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7"} err="failed to get container status \"50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7\": rpc error: code = NotFound desc = could not find container \"50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7\": container with ID starting with 50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7 not found: ID does not exist"
Sep 11 01:00:43.981593 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.981529    2574 scope.go:117] "RemoveContainer" containerID="aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3"
Sep 11 01:00:43.981766 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.981748    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3"} err="failed to get container status \"aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3\": rpc error: code = NotFound desc = could not find container \"aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3\": container with ID starting with aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3 not found: ID does not exist"
Sep 11 01:00:43.981810 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.981767    2574 scope.go:117] "RemoveContainer" containerID="c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768"
Sep 11 01:00:43.982003 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.981985    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768"} err="failed to get container status \"c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768\": rpc error: code = NotFound desc = could not find container \"c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768\": container with ID starting with c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768 not found: ID does not exist"
Sep 11 01:00:43.982047 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.982004    2574 scope.go:117] "RemoveContainer" containerID="0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79"
Sep 11 01:00:43.982213 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.982195    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79"} err="failed to get container status \"0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79\": rpc error: code = NotFound desc = could not find container \"0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79\": container with ID starting with 0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79 not found: ID does not exist"
Sep 11 01:00:43.982278 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.982214    2574 scope.go:117] "RemoveContainer" containerID="55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b"
Sep 11 01:00:43.982462 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.982445    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b"} err="failed to get container status \"55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b\": rpc error: code = NotFound desc = could not find container \"55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b\": container with ID starting with 55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b not found: ID does not exist"
Sep 11 01:00:43.982510 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.982464    2574 scope.go:117] "RemoveContainer" containerID="84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610"
Sep 11 01:00:43.982689 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.982672    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610"} err="failed to get container status \"84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610\": rpc error: code = NotFound desc = could not find container \"84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610\": container with ID starting with 84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610 not found: ID does not exist"
Sep 11 01:00:43.982749 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.982690    2574 scope.go:117] "RemoveContainer" containerID="736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5"
Sep 11 01:00:43.982931 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.982892    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5"} err="failed to get container status \"736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5\": rpc error: code = NotFound desc = could not find container \"736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5\": container with ID starting with 736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5 not found: ID does not exist"
Sep 11 01:00:43.982970 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.982952    2574 scope.go:117] "RemoveContainer" containerID="50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7"
Sep 11 01:00:43.983198 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.983183    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7"} err="failed to get container status \"50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7\": rpc error: code = NotFound desc = could not find container \"50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7\": container with ID starting with 50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7 not found: ID does not exist"
Sep 11 01:00:43.983251 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.983198    2574 scope.go:117] "RemoveContainer" containerID="aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3"
Sep 11 01:00:43.983402 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.983385    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3"} err="failed to get container status \"aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3\": rpc error: code = NotFound desc = could not find container \"aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3\": container with ID starting with aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3 not found: ID does not exist"
Sep 11 01:00:43.983466 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.983403    2574 scope.go:117] "RemoveContainer" containerID="c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768"
Sep 11 01:00:43.983610 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.983595    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768"} err="failed to get container status \"c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768\": rpc error: code = NotFound desc = could not find container \"c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768\": container with ID starting with c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768 not found: ID does not exist"
Sep 11 01:00:43.983655 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.983612    2574 scope.go:117] "RemoveContainer" containerID="0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79"
Sep 11 01:00:43.983805 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.983789    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79"} err="failed to get container status \"0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79\": rpc error: code = NotFound desc = could not find container \"0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79\": container with ID starting with 0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79 not found: ID does not exist"
Sep 11 01:00:43.983852 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.983805    2574 scope.go:117] "RemoveContainer" containerID="55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b"
Sep 11 01:00:43.984050 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.984033    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b"} err="failed to get container status \"55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b\": rpc error: code = NotFound desc = could not find container \"55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b\": container with ID starting with 55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b not found: ID does not exist"
Sep 11 01:00:43.984125 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.984050    2574 scope.go:117] "RemoveContainer" containerID="84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610"
Sep 11 01:00:43.984286 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.984270    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610"} err="failed to get container status \"84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610\": rpc error: code = NotFound desc = could not find container \"84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610\": container with ID starting with 84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610 not found: ID does not exist"
Sep 11 01:00:43.984350 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.984287    2574 scope.go:117] "RemoveContainer" containerID="736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5"
Sep 11 01:00:43.984482 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.984466    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5"} err="failed to get container status \"736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5\": rpc error: code = NotFound desc = could not find container \"736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5\": container with ID starting with 736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5 not found: ID does not exist"
Sep 11 01:00:43.984549 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.984482    2574 scope.go:117] "RemoveContainer" containerID="50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7"
Sep 11 01:00:43.984691 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.984673    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7"} err="failed to get container status \"50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7\": rpc error: code = NotFound desc = could not find container \"50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7\": container with ID starting with 50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7 not found: ID does not exist"
Sep 11 01:00:43.984750 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.984692    2574 scope.go:117] "RemoveContainer" containerID="aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3"
Sep 11 01:00:43.984900 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.984882    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3"} err="failed to get container status \"aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3\": rpc error: code = NotFound desc = could not find container \"aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3\": container with ID starting with aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3 not found: ID does not exist"
Sep 11 01:00:43.984973 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.984900    2574 scope.go:117] "RemoveContainer" containerID="c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768"
Sep 11 01:00:43.985153 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.985138    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768"} err="failed to get container status \"c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768\": rpc error: code = NotFound desc = could not find container \"c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768\": container with ID starting with c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768 not found: ID does not exist"
Sep 11 01:00:43.985201 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.985154    2574 scope.go:117] "RemoveContainer" containerID="0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79"
Sep 11 01:00:43.985351 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.985334    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79"} err="failed to get container status \"0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79\": rpc error: code = NotFound desc = could not find container \"0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79\": container with ID starting with 0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79 not found: ID does not exist"
Sep 11 01:00:43.985419 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.985364    2574 scope.go:117] "RemoveContainer" containerID="55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b"
Sep 11 01:00:43.985587 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.985572    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b"} err="failed to get container status \"55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b\": rpc error: code = NotFound desc = could not find container \"55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b\": container with ID starting with 55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b not found: ID does not exist"
Sep 11 01:00:43.985636 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.985587    2574 scope.go:117] "RemoveContainer" containerID="84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610"
Sep 11 01:00:43.985758 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.985744    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610"} err="failed to get container status \"84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610\": rpc error: code = NotFound desc = could not find container \"84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610\": container with ID starting with 84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610 not found: ID does not exist"
Sep 11 01:00:43.985803 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.985759    2574 scope.go:117] "RemoveContainer" containerID="736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5"
Sep 11 01:00:43.985972 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.985952    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5"} err="failed to get container status \"736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5\": rpc error: code = NotFound desc = could not find container \"736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5\": container with ID starting with 736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5 not found: ID does not exist"
Sep 11 01:00:43.986032 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.985973    2574 scope.go:117] "RemoveContainer" containerID="50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7"
Sep 11 01:00:43.986306 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.986280    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7"} err="failed to get container status \"50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7\": rpc error: code = NotFound desc = could not find container \"50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7\": container with ID starting with 50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7 not found: ID does not exist"
Sep 11 01:00:43.986404 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.986306    2574 scope.go:117] "RemoveContainer" containerID="aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3"
Sep 11 01:00:43.986690 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.986670    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3"} err="failed to get container status \"aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3\": rpc error: code = NotFound desc = could not find container \"aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3\": container with ID starting with aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3 not found: ID does not exist"
Sep 11 01:00:43.986690 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.986691    2574 scope.go:117] "RemoveContainer" containerID="c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768"
Sep 11 01:00:43.987003 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.986980    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768"} err="failed to get container status \"c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768\": rpc error: code = NotFound desc = could not find container \"c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768\": container with ID starting with c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768 not found: ID does not exist"
Sep 11 01:00:43.987003 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.987000    2574 scope.go:117] "RemoveContainer" containerID="0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79"
Sep 11 01:00:43.987253 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.987235    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79"} err="failed to get container status \"0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79\": rpc error: code = NotFound desc = could not find container \"0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79\": container with ID starting with 0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79 not found: ID does not exist"
Sep 11 01:00:43.987253 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.987252    2574 scope.go:117] "RemoveContainer" containerID="55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b"
Sep 11 01:00:43.987493 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.987472    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b"} err="failed to get container status \"55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b\": rpc error: code = NotFound desc = could not find container \"55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b\": container with ID starting with 55011376d417b6859140bcef5e384bed55520b27d1894329ce920652380a0e5b not found: ID does not exist"
Sep 11 01:00:43.987537 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.987495    2574 scope.go:117] "RemoveContainer" containerID="84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610"
Sep 11 01:00:43.987700 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.987682    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610"} err="failed to get container status \"84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610\": rpc error: code = NotFound desc = could not find container \"84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610\": container with ID starting with 84388ac033124469eca489b8f5b9b241869d650ed96e1ed765b7abaa610e1610 not found: ID does not exist"
Sep 11 01:00:43.987747 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.987700    2574 scope.go:117] "RemoveContainer" containerID="736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5"
Sep 11 01:00:43.987747 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.987725    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 01:00:43.987986 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.987965    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5"} err="failed to get container status \"736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5\": rpc error: code = NotFound desc = could not find container \"736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5\": container with ID starting with 736b5b68dd72ce3a2aae7da1f6929d080ab978db6a614a57ce9d0f37b7a29ab5 not found: ID does not exist"
Sep 11 01:00:43.988043 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.987988    2574 scope.go:117] "RemoveContainer" containerID="50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7"
Sep 11 01:00:43.988076 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.988043    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f8f5157-da76-4270-973d-bebc2d3b9e74" containerName="kube-rbac-proxy"
Sep 11 01:00:43.988076 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.988059    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f8f5157-da76-4270-973d-bebc2d3b9e74" containerName="kube-rbac-proxy"
Sep 11 01:00:43.988076 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.988071    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f8f5157-da76-4270-973d-bebc2d3b9e74" containerName="thanos-sidecar"
Sep 11 01:00:43.988076 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.988076    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f8f5157-da76-4270-973d-bebc2d3b9e74" containerName="thanos-sidecar"
Sep 11 01:00:43.988216 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.988082    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f8f5157-da76-4270-973d-bebc2d3b9e74" containerName="kube-rbac-proxy-thanos"
Sep 11 01:00:43.988216 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.988087    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f8f5157-da76-4270-973d-bebc2d3b9e74" containerName="kube-rbac-proxy-thanos"
Sep 11 01:00:43.988216 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.988093    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f8f5157-da76-4270-973d-bebc2d3b9e74" containerName="prometheus"
Sep 11 01:00:43.988216 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.988099    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f8f5157-da76-4270-973d-bebc2d3b9e74" containerName="prometheus"
Sep 11 01:00:43.988216 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.988112    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f8f5157-da76-4270-973d-bebc2d3b9e74" containerName="config-reloader"
Sep 11 01:00:43.988216 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.988120    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f8f5157-da76-4270-973d-bebc2d3b9e74" containerName="config-reloader"
Sep 11 01:00:43.988216 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.988128    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f8f5157-da76-4270-973d-bebc2d3b9e74" containerName="init-config-reloader"
Sep 11 01:00:43.988216 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.988132    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f8f5157-da76-4270-973d-bebc2d3b9e74" containerName="init-config-reloader"
Sep 11 01:00:43.988216 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.988137    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f8f5157-da76-4270-973d-bebc2d3b9e74" containerName="kube-rbac-proxy-web"
Sep 11 01:00:43.988216 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.988142    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f8f5157-da76-4270-973d-bebc2d3b9e74" containerName="kube-rbac-proxy-web"
Sep 11 01:00:43.988216 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.988186    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f8f5157-da76-4270-973d-bebc2d3b9e74" containerName="thanos-sidecar"
Sep 11 01:00:43.988216 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.988194    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f8f5157-da76-4270-973d-bebc2d3b9e74" containerName="prometheus"
Sep 11 01:00:43.988216 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.988192    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7"} err="failed to get container status \"50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7\": rpc error: code = NotFound desc = could not find container \"50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7\": container with ID starting with 50eecfdae0b4b512fddce5594a5c841c55d4d29587e7ec8b7d90a51ef85b47c7 not found: ID does not exist"
Sep 11 01:00:43.988216 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.988201    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f8f5157-da76-4270-973d-bebc2d3b9e74" containerName="config-reloader"
Sep 11 01:00:43.988216 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.988205    2574 scope.go:117] "RemoveContainer" containerID="aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3"
Sep 11 01:00:43.988706 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.988207    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f8f5157-da76-4270-973d-bebc2d3b9e74" containerName="kube-rbac-proxy-web"
Sep 11 01:00:43.988706 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.988252    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f8f5157-da76-4270-973d-bebc2d3b9e74" containerName="kube-rbac-proxy"
Sep 11 01:00:43.988706 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.988260    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f8f5157-da76-4270-973d-bebc2d3b9e74" containerName="kube-rbac-proxy-thanos"
Sep 11 01:00:43.988706 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.988439    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3"} err="failed to get container status \"aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3\": rpc error: code = NotFound desc = could not find container \"aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3\": container with ID starting with aacf8287d55b52249b1e019cc70728ffebd3ce40947ae5ded3a1c8688ea0fcc3 not found: ID does not exist"
Sep 11 01:00:43.988706 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.988456    2574 scope.go:117] "RemoveContainer" containerID="c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768"
Sep 11 01:00:43.988706 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.988668    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768"} err="failed to get container status \"c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768\": rpc error: code = NotFound desc = could not find container \"c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768\": container with ID starting with c5a4e991ee44f22eaaabe6d51837d2ed974c834b147a20d14cd1519783db2768 not found: ID does not exist"
Sep 11 01:00:43.988706 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.988687    2574 scope.go:117] "RemoveContainer" containerID="0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79"
Sep 11 01:00:43.988933 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.988885    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79"} err="failed to get container status \"0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79\": rpc error: code = NotFound desc = could not find container \"0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79\": container with ID starting with 0ac6ddfbd30b81090b2261800570447ea8baf5d931a67906b1ed2557cf4d0c79 not found: ID does not exist"
Sep 11 01:00:43.991243 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.991229    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:43.993829 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.993811    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 11 01:00:43.993950 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.993867    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 11 01:00:43.993950 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.993883    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 11 01:00:43.993950 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.993940    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 11 01:00:43.994096 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.993902    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-mai91c4a443g\""
Sep 11 01:00:43.994340 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.994324    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 11 01:00:43.994409 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.994335    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 11 01:00:43.994457 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.994440    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 11 01:00:43.994786 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.994768    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 11 01:00:43.994879 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.994788    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 11 01:00:43.994879 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.994831    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 11 01:00:43.994879 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.994867    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-h8mht\""
Sep 11 01:00:43.995158 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.995143    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 11 01:00:43.995158 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.995148    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 11 01:00:43.996768 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:43.996748    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 11 01:00:44.001120 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.001099    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 11 01:00:44.011460 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.011440    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 01:00:44.135487 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.135397    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e83f749b-c698-4e1b-a415-5a532db7da0b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.135487 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.135437    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e83f749b-c698-4e1b-a415-5a532db7da0b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.135487 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.135456    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e83f749b-c698-4e1b-a415-5a532db7da0b-web-config\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.135487 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.135477    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e83f749b-c698-4e1b-a415-5a532db7da0b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.135725 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.135521    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e83f749b-c698-4e1b-a415-5a532db7da0b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.135725 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.135546    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e83f749b-c698-4e1b-a415-5a532db7da0b-config\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.135725 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.135579    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e83f749b-c698-4e1b-a415-5a532db7da0b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.135725 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.135594    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e83f749b-c698-4e1b-a415-5a532db7da0b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.135725 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.135608    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e83f749b-c698-4e1b-a415-5a532db7da0b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.135725 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.135629    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e83f749b-c698-4e1b-a415-5a532db7da0b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.135725 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.135655    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e83f749b-c698-4e1b-a415-5a532db7da0b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.135725 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.135670    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e83f749b-c698-4e1b-a415-5a532db7da0b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.135725 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.135687    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e83f749b-c698-4e1b-a415-5a532db7da0b-config-out\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.135725 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.135703    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e83f749b-c698-4e1b-a415-5a532db7da0b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.135725 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.135725    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e83f749b-c698-4e1b-a415-5a532db7da0b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.136069 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.135743    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bl4m\" (UniqueName: \"kubernetes.io/projected/e83f749b-c698-4e1b-a415-5a532db7da0b-kube-api-access-8bl4m\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.136069 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.135759    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e83f749b-c698-4e1b-a415-5a532db7da0b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.136069 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.135801    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e83f749b-c698-4e1b-a415-5a532db7da0b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.136069 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.135825    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e83f749b-c698-4e1b-a415-5a532db7da0b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.136069 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.135844    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e83f749b-c698-4e1b-a415-5a532db7da0b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.236716 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.236684    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e83f749b-c698-4e1b-a415-5a532db7da0b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.236842 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.236721    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e83f749b-c698-4e1b-a415-5a532db7da0b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.236842 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.236741    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e83f749b-c698-4e1b-a415-5a532db7da0b-web-config\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.236842 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.236760    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e83f749b-c698-4e1b-a415-5a532db7da0b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.236842 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.236776    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e83f749b-c698-4e1b-a415-5a532db7da0b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.237028 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.236983    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e83f749b-c698-4e1b-a415-5a532db7da0b-config\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.237080 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.237041    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e83f749b-c698-4e1b-a415-5a532db7da0b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.237080 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.237069    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e83f749b-c698-4e1b-a415-5a532db7da0b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.237493 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.237462    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e83f749b-c698-4e1b-a415-5a532db7da0b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.237603 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.237531    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e83f749b-c698-4e1b-a415-5a532db7da0b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.237735 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.237671    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e83f749b-c698-4e1b-a415-5a532db7da0b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.237735 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.237716    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e83f749b-c698-4e1b-a415-5a532db7da0b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.238296 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.238268    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e83f749b-c698-4e1b-a415-5a532db7da0b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.238296 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.238288    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e83f749b-c698-4e1b-a415-5a532db7da0b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.238443 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.238341    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e83f749b-c698-4e1b-a415-5a532db7da0b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.238443 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.238374    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e83f749b-c698-4e1b-a415-5a532db7da0b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.238443 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.238404    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e83f749b-c698-4e1b-a415-5a532db7da0b-config-out\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.238443 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.238434    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e83f749b-c698-4e1b-a415-5a532db7da0b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.238652 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.238457    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e83f749b-c698-4e1b-a415-5a532db7da0b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.238652 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.238492    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bl4m\" (UniqueName: \"kubernetes.io/projected/e83f749b-c698-4e1b-a415-5a532db7da0b-kube-api-access-8bl4m\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.238652 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.238517    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e83f749b-c698-4e1b-a415-5a532db7da0b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.238652 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.238545    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e83f749b-c698-4e1b-a415-5a532db7da0b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.238652 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.238569    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e83f749b-c698-4e1b-a415-5a532db7da0b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.238652 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.238596    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e83f749b-c698-4e1b-a415-5a532db7da0b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.239051 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.239028    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e83f749b-c698-4e1b-a415-5a532db7da0b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.239457 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.239428    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e83f749b-c698-4e1b-a415-5a532db7da0b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.240300 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.240019    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e83f749b-c698-4e1b-a415-5a532db7da0b-web-config\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.240300 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.240172    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e83f749b-c698-4e1b-a415-5a532db7da0b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.240300 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.240220    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e83f749b-c698-4e1b-a415-5a532db7da0b-config\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.240300 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.240255    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e83f749b-c698-4e1b-a415-5a532db7da0b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.241110 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.240789    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e83f749b-c698-4e1b-a415-5a532db7da0b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.241110 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.241027    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e83f749b-c698-4e1b-a415-5a532db7da0b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.241334 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.241300    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e83f749b-c698-4e1b-a415-5a532db7da0b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.242012 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.241988    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e83f749b-c698-4e1b-a415-5a532db7da0b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.242361 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.242337    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e83f749b-c698-4e1b-a415-5a532db7da0b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.242727 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.242700    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e83f749b-c698-4e1b-a415-5a532db7da0b-config-out\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.242727 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.242713    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e83f749b-c698-4e1b-a415-5a532db7da0b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.242824 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.242774    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e83f749b-c698-4e1b-a415-5a532db7da0b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.243165 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.243147    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e83f749b-c698-4e1b-a415-5a532db7da0b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.247414 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.247396    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bl4m\" (UniqueName: \"kubernetes.io/projected/e83f749b-c698-4e1b-a415-5a532db7da0b-kube-api-access-8bl4m\") pod \"prometheus-k8s-0\" (UID: \"e83f749b-c698-4e1b-a415-5a532db7da0b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.305363 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.305342    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:44.586823 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.586687    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 01:00:44.589277 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:00:44.589250    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode83f749b_c698_4e1b_a415_5a532db7da0b.slice/crio-767a70b9f269f95670587cd816f3ca6a1ed8bafc6f10bf07f0fdcec34a556917 WatchSource:0}: Error finding container 767a70b9f269f95670587cd816f3ca6a1ed8bafc6f10bf07f0fdcec34a556917: Status 404 returned error can't find the container with id 767a70b9f269f95670587cd816f3ca6a1ed8bafc6f10bf07f0fdcec34a556917
Sep 11 01:00:44.933807 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.933775    2574 generic.go:356] "Generic (PLEG): container finished" podID="e83f749b-c698-4e1b-a415-5a532db7da0b" containerID="a1125d81296031dc6e04a3b15a1290f4889f77cf837ec7605fb1f75b83000f89" exitCode=0
Sep 11 01:00:44.934000 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.933818    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e83f749b-c698-4e1b-a415-5a532db7da0b","Type":"ContainerDied","Data":"a1125d81296031dc6e04a3b15a1290f4889f77cf837ec7605fb1f75b83000f89"}
Sep 11 01:00:44.934000 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:44.933882    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e83f749b-c698-4e1b-a415-5a532db7da0b","Type":"ContainerStarted","Data":"767a70b9f269f95670587cd816f3ca6a1ed8bafc6f10bf07f0fdcec34a556917"}
Sep 11 01:00:45.102196 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:45.102172    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f8f5157-da76-4270-973d-bebc2d3b9e74" path="/var/lib/kubelet/pods/7f8f5157-da76-4270-973d-bebc2d3b9e74/volumes"
Sep 11 01:00:45.940991 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:45.940959    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e83f749b-c698-4e1b-a415-5a532db7da0b","Type":"ContainerStarted","Data":"784ef5b6bf7fb13af56cf540f04d0ffd409768c274d04abdd85096800680a3af"}
Sep 11 01:00:45.941326 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:45.941045    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e83f749b-c698-4e1b-a415-5a532db7da0b","Type":"ContainerStarted","Data":"426d8d52157d4c10bccfd82f627a374a4e6f98c999c49946494bffc4c2385089"}
Sep 11 01:00:45.941326 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:45.941083    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e83f749b-c698-4e1b-a415-5a532db7da0b","Type":"ContainerStarted","Data":"b66d7932a7f89119f644db23d77571f0761f26c8da5994db3ebd0dd204f36508"}
Sep 11 01:00:45.941326 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:45.941109    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e83f749b-c698-4e1b-a415-5a532db7da0b","Type":"ContainerStarted","Data":"802c1c308a75511d0e46b620c2117c2f49eb7e2c890f625819f8b29774929f33"}
Sep 11 01:00:45.941326 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:45.941132    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e83f749b-c698-4e1b-a415-5a532db7da0b","Type":"ContainerStarted","Data":"4c438cac36a7b84574d6d71139fa5d50cde02575fd7dd76e52d6983c0887b6e6"}
Sep 11 01:00:45.941326 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:45.941154    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e83f749b-c698-4e1b-a415-5a532db7da0b","Type":"ContainerStarted","Data":"2a246762cf6c1183663bce286690907e1fcc210c4fab8501222e21315f6c8198"}
Sep 11 01:00:45.976262 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:45.976194    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.957326346 podStartE2EDuration="2.976173338s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.846992ms" isStatefulPod=false podCreationTimestamp="2026-09-11 01:00:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:00:45.974425721 +0000 UTC m=+161.371617894" watchObservedRunningTime="2026-09-11 01:00:45.976173338 +0000 UTC m=+161.373365511"
Sep 11 01:00:46.172028 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:46.171992    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:00:49.306485 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:49.306438    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:00:57.185440 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:57.185354    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:00:57.286008 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:57.285981    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:00:57.337390 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:00:57.337359    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:01:28.844158 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:01:28.844124    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:01:44.306243 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:01:44.306208    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:01:44.323250 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:01:44.323219    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:01:45.160606 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:01:45.160580    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 01:01:53.338817 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:01:53.338785    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:01:53.391964 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:01:53.391934    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:01:56.943160 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:01:56.943122    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:01:57.012160 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:01:57.012124    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:02:05.315530 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:02:05.315491    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:02:05.370223 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:02:05.370189    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:02:07.409657 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:02:07.409618    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:02:07.508780 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:02:07.508742    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:02:07.608563 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:02:07.608533    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:02:20.764384 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:02:20.764331    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:02:20.863517 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:02:20.863480    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:03:00.546474 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:00.546434    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:03:01.844950 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:01.844896    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:03:05.003143 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:05.003114    2574 kubelet.go:1648] "Image garbage collection succeeded"
Sep 11 01:03:27.334645 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:27.334609    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:03:28.313872 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:28.313841    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:03:31.718156 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:31.718118    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:03:31.817896 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:31.817865    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:03:33.121325 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:33.121294    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:03:33.242433 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:33.242399    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:03:33.809958 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:33.809904    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:03:33.910371 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:33.910337    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:03:36.087234 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:36.087165    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:03:36.246178 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:36.246144    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:03:37.537267 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:37.537206    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:03:37.637559 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:37.637528    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:03:42.974846 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:42.974790    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:03:43.029203 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:43.029173    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:03:44.215102 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:44.215057    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:03:44.383977 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:44.383931    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:03:44.483954 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:44.483806    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:03:46.244939 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:46.244892    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rssq5"]
Sep 11 01:03:46.247882 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:46.247866    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rssq5"
Sep 11 01:03:46.250988 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:46.250968    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 11 01:03:46.251104 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:46.250968    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-b4h2k\""
Sep 11 01:03:46.251104 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:46.251005    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 11 01:03:46.256202 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:46.256180    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rssq5"]
Sep 11 01:03:46.351330 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:46.351298    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdj28\" (UniqueName: \"kubernetes.io/projected/57ea4d22-0d1f-49b4-84f4-dab05cc44f84-kube-api-access-wdj28\") pod \"openshift-lws-operator-fd8ccff4c-rssq5\" (UID: \"57ea4d22-0d1f-49b4-84f4-dab05cc44f84\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rssq5"
Sep 11 01:03:46.351503 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:46.351350    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/57ea4d22-0d1f-49b4-84f4-dab05cc44f84-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rssq5\" (UID: \"57ea4d22-0d1f-49b4-84f4-dab05cc44f84\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rssq5"
Sep 11 01:03:46.452200 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:46.452166    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/57ea4d22-0d1f-49b4-84f4-dab05cc44f84-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rssq5\" (UID: \"57ea4d22-0d1f-49b4-84f4-dab05cc44f84\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rssq5"
Sep 11 01:03:46.452382 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:46.452217    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdj28\" (UniqueName: \"kubernetes.io/projected/57ea4d22-0d1f-49b4-84f4-dab05cc44f84-kube-api-access-wdj28\") pod \"openshift-lws-operator-fd8ccff4c-rssq5\" (UID: \"57ea4d22-0d1f-49b4-84f4-dab05cc44f84\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rssq5"
Sep 11 01:03:46.452631 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:46.452606    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/57ea4d22-0d1f-49b4-84f4-dab05cc44f84-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rssq5\" (UID: \"57ea4d22-0d1f-49b4-84f4-dab05cc44f84\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rssq5"
Sep 11 01:03:46.463387 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:46.463358    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdj28\" (UniqueName: \"kubernetes.io/projected/57ea4d22-0d1f-49b4-84f4-dab05cc44f84-kube-api-access-wdj28\") pod \"openshift-lws-operator-fd8ccff4c-rssq5\" (UID: \"57ea4d22-0d1f-49b4-84f4-dab05cc44f84\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rssq5"
Sep 11 01:03:46.565146 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:46.565065    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rssq5"
Sep 11 01:03:46.841715 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:46.841644    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rssq5"]
Sep 11 01:03:46.845737 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:03:46.845710    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod57ea4d22_0d1f_49b4_84f4_dab05cc44f84.slice/crio-16d910bbcf327bae37198df35a4b3d63c39a1e895fcc9c77fa7b546f3f69b427 WatchSource:0}: Error finding container 16d910bbcf327bae37198df35a4b3d63c39a1e895fcc9c77fa7b546f3f69b427: Status 404 returned error can't find the container with id 16d910bbcf327bae37198df35a4b3d63c39a1e895fcc9c77fa7b546f3f69b427
Sep 11 01:03:46.847755 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:46.847736    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 01:03:47.575533 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:47.575498    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rssq5" event={"ID":"57ea4d22-0d1f-49b4-84f4-dab05cc44f84","Type":"ContainerStarted","Data":"16d910bbcf327bae37198df35a4b3d63c39a1e895fcc9c77fa7b546f3f69b427"}
Sep 11 01:03:50.588518 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:50.588479    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rssq5" event={"ID":"57ea4d22-0d1f-49b4-84f4-dab05cc44f84","Type":"ContainerStarted","Data":"48dfd16b320a7cdca7783cd9bb1fdd58d17c5a555436d715b7c1813f9543abad"}
Sep 11 01:03:50.608730 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:50.608682    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rssq5" podStartSLOduration=1.5559957020000001 podStartE2EDuration="4.608668752s" totalImagesPullingTime="3.05267305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:03:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:03:50.607828697 +0000 UTC m=+346.005020870" watchObservedRunningTime="2026-09-11 01:03:50.608668752 +0000 UTC m=+346.005860925"
Sep 11 01:03:51.407822 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:51.407781    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:03:51.510387 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:51.510359    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:03:53.575887 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:03:53.575858    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:09.051573 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:09.051493    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:09.167416 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:09.167378    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:09.761829 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:09.761793    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:10.011445 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:10.011415    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:10.116224 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:10.116136    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:10.188451 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:10.188424    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-nvrtp"]
Sep 11 01:04:10.191654 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:10.191636    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-nvrtp"
Sep 11 01:04:10.198308 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:10.198287    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 11 01:04:10.198874 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:10.198858    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 11 01:04:10.199883 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:10.199864    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-h5l9p\""
Sep 11 01:04:10.200155 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:10.200136    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 11 01:04:10.200237 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:10.200189    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 11 01:04:10.206969 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:10.206944    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-nvrtp"]
Sep 11 01:04:10.347403 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:10.347372    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/86bf8c2c-cd19-484f-8b03-659d42056d67-webhook-cert\") pod \"opendatahub-operator-controller-manager-8bd47c7d5-nvrtp\" (UID: \"86bf8c2c-cd19-484f-8b03-659d42056d67\") " pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-nvrtp"
Sep 11 01:04:10.347403 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:10.347409    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86bf8c2c-cd19-484f-8b03-659d42056d67-tmp\") pod \"opendatahub-operator-controller-manager-8bd47c7d5-nvrtp\" (UID: \"86bf8c2c-cd19-484f-8b03-659d42056d67\") " pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-nvrtp"
Sep 11 01:04:10.347631 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:10.347435    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/86bf8c2c-cd19-484f-8b03-659d42056d67-manifests\") pod \"opendatahub-operator-controller-manager-8bd47c7d5-nvrtp\" (UID: \"86bf8c2c-cd19-484f-8b03-659d42056d67\") " pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-nvrtp"
Sep 11 01:04:10.347631 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:10.347488    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/86bf8c2c-cd19-484f-8b03-659d42056d67-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8bd47c7d5-nvrtp\" (UID: \"86bf8c2c-cd19-484f-8b03-659d42056d67\") " pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-nvrtp"
Sep 11 01:04:10.347631 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:10.347537    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl45g\" (UniqueName: \"kubernetes.io/projected/86bf8c2c-cd19-484f-8b03-659d42056d67-kube-api-access-tl45g\") pod \"opendatahub-operator-controller-manager-8bd47c7d5-nvrtp\" (UID: \"86bf8c2c-cd19-484f-8b03-659d42056d67\") " pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-nvrtp"
Sep 11 01:04:10.352671 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:10.352641    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:10.445402 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:10.445372    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:10.448581 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:10.448563    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/86bf8c2c-cd19-484f-8b03-659d42056d67-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8bd47c7d5-nvrtp\" (UID: \"86bf8c2c-cd19-484f-8b03-659d42056d67\") " pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-nvrtp"
Sep 11 01:04:10.448652 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:10.448603    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl45g\" (UniqueName: \"kubernetes.io/projected/86bf8c2c-cd19-484f-8b03-659d42056d67-kube-api-access-tl45g\") pod \"opendatahub-operator-controller-manager-8bd47c7d5-nvrtp\" (UID: \"86bf8c2c-cd19-484f-8b03-659d42056d67\") " pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-nvrtp"
Sep 11 01:04:10.448712 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:10.448654    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/86bf8c2c-cd19-484f-8b03-659d42056d67-webhook-cert\") pod \"opendatahub-operator-controller-manager-8bd47c7d5-nvrtp\" (UID: \"86bf8c2c-cd19-484f-8b03-659d42056d67\") " pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-nvrtp"
Sep 11 01:04:10.448712 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:10.448676    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86bf8c2c-cd19-484f-8b03-659d42056d67-tmp\") pod \"opendatahub-operator-controller-manager-8bd47c7d5-nvrtp\" (UID: \"86bf8c2c-cd19-484f-8b03-659d42056d67\") " pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-nvrtp"
Sep 11 01:04:10.448808 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:10.448740    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/86bf8c2c-cd19-484f-8b03-659d42056d67-manifests\") pod \"opendatahub-operator-controller-manager-8bd47c7d5-nvrtp\" (UID: \"86bf8c2c-cd19-484f-8b03-659d42056d67\") " pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-nvrtp"
Sep 11 01:04:10.449134 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:10.449107    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/86bf8c2c-cd19-484f-8b03-659d42056d67-manifests\") pod \"opendatahub-operator-controller-manager-8bd47c7d5-nvrtp\" (UID: \"86bf8c2c-cd19-484f-8b03-659d42056d67\") " pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-nvrtp"
Sep 11 01:04:10.451021 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:10.450991    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86bf8c2c-cd19-484f-8b03-659d42056d67-tmp\") pod \"opendatahub-operator-controller-manager-8bd47c7d5-nvrtp\" (UID: \"86bf8c2c-cd19-484f-8b03-659d42056d67\") " pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-nvrtp"
Sep 11 01:04:10.451268 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:10.451249    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/86bf8c2c-cd19-484f-8b03-659d42056d67-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8bd47c7d5-nvrtp\" (UID: \"86bf8c2c-cd19-484f-8b03-659d42056d67\") " pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-nvrtp"
Sep 11 01:04:10.451366 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:10.451345    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/86bf8c2c-cd19-484f-8b03-659d42056d67-webhook-cert\") pod \"opendatahub-operator-controller-manager-8bd47c7d5-nvrtp\" (UID: \"86bf8c2c-cd19-484f-8b03-659d42056d67\") " pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-nvrtp"
Sep 11 01:04:10.460034 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:10.460011    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl45g\" (UniqueName: \"kubernetes.io/projected/86bf8c2c-cd19-484f-8b03-659d42056d67-kube-api-access-tl45g\") pod \"opendatahub-operator-controller-manager-8bd47c7d5-nvrtp\" (UID: \"86bf8c2c-cd19-484f-8b03-659d42056d67\") " pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-nvrtp"
Sep 11 01:04:10.501774 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:10.501743    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-nvrtp"
Sep 11 01:04:10.545375 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:10.545339    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:10.652568 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:10.652531    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:10.747575 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:10.747497    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:10.799138 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:10.799108    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-nvrtp"]
Sep 11 01:04:11.538012 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:11.537969    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:11.650194 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:11.650142    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:11.666196 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:11.666164    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-nvrtp" event={"ID":"86bf8c2c-cd19-484f-8b03-659d42056d67","Type":"ContainerStarted","Data":"9fb46ea573b0c2efadd405038638188b214d1d71825f89ebeec1835a2553560c"}
Sep 11 01:04:11.738618 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:11.738583    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:12.282700 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:12.282658    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:12.382661 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:12.382623    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:14.680498 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:14.680462    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-nvrtp" event={"ID":"86bf8c2c-cd19-484f-8b03-659d42056d67","Type":"ContainerStarted","Data":"bd268ff3d462ff433df38c44b1eb86e5b3b97d67a791ba128a5d5b55efdc18c0"}
Sep 11 01:04:15.685480 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:15.685444    2574 generic.go:356] "Generic (PLEG): container finished" podID="86bf8c2c-cd19-484f-8b03-659d42056d67" containerID="bd268ff3d462ff433df38c44b1eb86e5b3b97d67a791ba128a5d5b55efdc18c0" exitCode=0
Sep 11 01:04:15.685858 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:15.685531    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-nvrtp" event={"ID":"86bf8c2c-cd19-484f-8b03-659d42056d67","Type":"ContainerDied","Data":"bd268ff3d462ff433df38c44b1eb86e5b3b97d67a791ba128a5d5b55efdc18c0"}
Sep 11 01:04:16.690840 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:16.690802    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-nvrtp" event={"ID":"86bf8c2c-cd19-484f-8b03-659d42056d67","Type":"ContainerStarted","Data":"ff67a809b15a4c61a31ca0da5a7254f5f01df81fd4fdd8fd0d1c38f0cf439208"}
Sep 11 01:04:16.691248 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:16.690955    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-nvrtp"
Sep 11 01:04:16.712121 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:16.712073    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-nvrtp" podStartSLOduration=2.329955256 podStartE2EDuration="6.712057775s" totalImagesPullingTime="3.37584376s" totalInitContainerRuntime="1.006258759s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:04:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:04:16.71026858 +0000 UTC m=+372.107460744" watchObservedRunningTime="2026-09-11 01:04:16.712057775 +0000 UTC m=+372.109249949"
Sep 11 01:04:18.098010 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:18.097971    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:18.198774 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:18.198738    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:22.104369 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:22.104330    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:22.206166 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:22.206130    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:22.261983 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:22.261955    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:27.697087 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:27.697052    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-nvrtp"
Sep 11 01:04:29.325994 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:29.325961    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:29.479666 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:29.479631    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:29.757704 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:29.757666    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:29.760342 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:29.760317    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-58fc6d6988-dvf8m"]
Sep 11 01:04:29.763474 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:29.763455    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-58fc6d6988-dvf8m"
Sep 11 01:04:29.771433 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:29.771306    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 11 01:04:29.771433 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:29.771320    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 11 01:04:29.771433 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:29.771412    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 11 01:04:29.772135 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:29.772118    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-k8rk7\""
Sep 11 01:04:29.772246 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:29.772144    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 11 01:04:29.785232 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:29.785200    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-58fc6d6988-dvf8m"]
Sep 11 01:04:29.813197 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:29.813174    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8483ab9d-a863-4d4b-93bd-8ca7d451c493-tmp\") pod \"kube-auth-proxy-58fc6d6988-dvf8m\" (UID: \"8483ab9d-a863-4d4b-93bd-8ca7d451c493\") " pod="openshift-ingress/kube-auth-proxy-58fc6d6988-dvf8m"
Sep 11 01:04:29.813300 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:29.813235    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8483ab9d-a863-4d4b-93bd-8ca7d451c493-tls-certs\") pod \"kube-auth-proxy-58fc6d6988-dvf8m\" (UID: \"8483ab9d-a863-4d4b-93bd-8ca7d451c493\") " pod="openshift-ingress/kube-auth-proxy-58fc6d6988-dvf8m"
Sep 11 01:04:29.813359 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:29.813341    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wmzc\" (UniqueName: \"kubernetes.io/projected/8483ab9d-a863-4d4b-93bd-8ca7d451c493-kube-api-access-6wmzc\") pod \"kube-auth-proxy-58fc6d6988-dvf8m\" (UID: \"8483ab9d-a863-4d4b-93bd-8ca7d451c493\") " pod="openshift-ingress/kube-auth-proxy-58fc6d6988-dvf8m"
Sep 11 01:04:29.913854 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:29.913822    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8483ab9d-a863-4d4b-93bd-8ca7d451c493-tls-certs\") pod \"kube-auth-proxy-58fc6d6988-dvf8m\" (UID: \"8483ab9d-a863-4d4b-93bd-8ca7d451c493\") " pod="openshift-ingress/kube-auth-proxy-58fc6d6988-dvf8m"
Sep 11 01:04:29.914025 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:29.913884    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wmzc\" (UniqueName: \"kubernetes.io/projected/8483ab9d-a863-4d4b-93bd-8ca7d451c493-kube-api-access-6wmzc\") pod \"kube-auth-proxy-58fc6d6988-dvf8m\" (UID: \"8483ab9d-a863-4d4b-93bd-8ca7d451c493\") " pod="openshift-ingress/kube-auth-proxy-58fc6d6988-dvf8m"
Sep 11 01:04:29.914025 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:29.913940    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8483ab9d-a863-4d4b-93bd-8ca7d451c493-tmp\") pod \"kube-auth-proxy-58fc6d6988-dvf8m\" (UID: \"8483ab9d-a863-4d4b-93bd-8ca7d451c493\") " pod="openshift-ingress/kube-auth-proxy-58fc6d6988-dvf8m"
Sep 11 01:04:29.916374 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:29.916341    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8483ab9d-a863-4d4b-93bd-8ca7d451c493-tmp\") pod \"kube-auth-proxy-58fc6d6988-dvf8m\" (UID: \"8483ab9d-a863-4d4b-93bd-8ca7d451c493\") " pod="openshift-ingress/kube-auth-proxy-58fc6d6988-dvf8m"
Sep 11 01:04:29.916562 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:29.916545    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8483ab9d-a863-4d4b-93bd-8ca7d451c493-tls-certs\") pod \"kube-auth-proxy-58fc6d6988-dvf8m\" (UID: \"8483ab9d-a863-4d4b-93bd-8ca7d451c493\") " pod="openshift-ingress/kube-auth-proxy-58fc6d6988-dvf8m"
Sep 11 01:04:29.943333 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:29.943310    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wmzc\" (UniqueName: \"kubernetes.io/projected/8483ab9d-a863-4d4b-93bd-8ca7d451c493-kube-api-access-6wmzc\") pod \"kube-auth-proxy-58fc6d6988-dvf8m\" (UID: \"8483ab9d-a863-4d4b-93bd-8ca7d451c493\") " pod="openshift-ingress/kube-auth-proxy-58fc6d6988-dvf8m"
Sep 11 01:04:30.062606 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:30.062513    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:30.073650 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:30.073632    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-58fc6d6988-dvf8m"
Sep 11 01:04:30.350946 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:30.350854    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-58fc6d6988-dvf8m"]
Sep 11 01:04:30.353487 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:04:30.353465    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8483ab9d_a863_4d4b_93bd_8ca7d451c493.slice/crio-29cc783c6c94f3e0c2f2f864b98f103bf75936134e2d4eb5d8a8a9e6737bca9d WatchSource:0}: Error finding container 29cc783c6c94f3e0c2f2f864b98f103bf75936134e2d4eb5d8a8a9e6737bca9d: Status 404 returned error can't find the container with id 29cc783c6c94f3e0c2f2f864b98f103bf75936134e2d4eb5d8a8a9e6737bca9d
Sep 11 01:04:30.742644 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:30.742613    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-58fc6d6988-dvf8m" event={"ID":"8483ab9d-a863-4d4b-93bd-8ca7d451c493","Type":"ContainerStarted","Data":"29cc783c6c94f3e0c2f2f864b98f103bf75936134e2d4eb5d8a8a9e6737bca9d"}
Sep 11 01:04:31.210359 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:31.210326    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p"]
Sep 11 01:04:31.213946 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:31.213894    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p"
Sep 11 01:04:31.218100 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:31.218036    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 11 01:04:31.218100 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:31.218050    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 11 01:04:31.218100 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:31.218036    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 11 01:04:31.218685 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:31.218645    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 11 01:04:31.218844 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:31.218821    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-ck7ll\""
Sep 11 01:04:31.223547 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:31.223429    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-vwx5p\" (UID: \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p"
Sep 11 01:04:31.223547 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:31.223483    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-vwx5p\" (UID: \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p"
Sep 11 01:04:31.223667 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:31.223566    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-vwx5p\" (UID: \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p"
Sep 11 01:04:31.223667 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:31.223656    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-vwx5p\" (UID: \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p"
Sep 11 01:04:31.223771 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:31.223683    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-vwx5p\" (UID: \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p"
Sep 11 01:04:31.223771 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:31.223709    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjrtp\" (UniqueName: \"kubernetes.io/projected/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-kube-api-access-bjrtp\") pod \"istiod-openshift-gateway-7b7995d5cd-vwx5p\" (UID: \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p"
Sep 11 01:04:31.223867 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:31.223787    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-vwx5p\" (UID: \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p"
Sep 11 01:04:31.229715 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:31.229617    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p"]
Sep 11 01:04:31.231832 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:31.231765    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:31.318256 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:31.318220    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:31.324515 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:31.324490    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-vwx5p\" (UID: \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p"
Sep 11 01:04:31.324643 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:31.324522    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-vwx5p\" (UID: \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p"
Sep 11 01:04:31.324643 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:31.324537    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjrtp\" (UniqueName: \"kubernetes.io/projected/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-kube-api-access-bjrtp\") pod \"istiod-openshift-gateway-7b7995d5cd-vwx5p\" (UID: \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p"
Sep 11 01:04:31.324643 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:31.324565    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-vwx5p\" (UID: \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p"
Sep 11 01:04:31.324643 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:31.324604    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-vwx5p\" (UID: \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p"
Sep 11 01:04:31.324643 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:31.324643    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-vwx5p\" (UID: \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p"
Sep 11 01:04:31.324898 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:31.324678    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-vwx5p\" (UID: \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p"
Sep 11 01:04:31.325613 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:31.325560    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-vwx5p\" (UID: \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p"
Sep 11 01:04:31.327476 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:31.327432    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-vwx5p\" (UID: \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p"
Sep 11 01:04:31.327599 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:31.327535    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-vwx5p\" (UID: \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p"
Sep 11 01:04:31.327599 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:31.327571    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-vwx5p\" (UID: \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p"
Sep 11 01:04:31.327768 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:31.327744    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-vwx5p\" (UID: \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p"
Sep 11 01:04:31.334319 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:31.334276    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjrtp\" (UniqueName: \"kubernetes.io/projected/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-kube-api-access-bjrtp\") pod \"istiod-openshift-gateway-7b7995d5cd-vwx5p\" (UID: \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p"
Sep 11 01:04:31.336074 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:31.336050    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-vwx5p\" (UID: \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p"
Sep 11 01:04:31.526859 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:31.526760    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p"
Sep 11 01:04:31.817601 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:31.817518    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p"]
Sep 11 01:04:32.325135 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:32.325097    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:33.136828 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:04:33.136793    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8ca41d9_7ec4_4344_a9d5_b7d10fdb34e5.slice/crio-03063b10150e48355f698afe08c2b9713bcfebcb94f2a69ad0381be1146181ed WatchSource:0}: Error finding container 03063b10150e48355f698afe08c2b9713bcfebcb94f2a69ad0381be1146181ed: Status 404 returned error can't find the container with id 03063b10150e48355f698afe08c2b9713bcfebcb94f2a69ad0381be1146181ed
Sep 11 01:04:33.758581 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:33.758545    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-58fc6d6988-dvf8m" event={"ID":"8483ab9d-a863-4d4b-93bd-8ca7d451c493","Type":"ContainerStarted","Data":"25fb2ba8c82b1f7e99293cc68c48ce999b7bf1d384b278d94575b381505ca0c8"}
Sep 11 01:04:33.759765 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:33.759743    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p" event={"ID":"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5","Type":"ContainerStarted","Data":"03063b10150e48355f698afe08c2b9713bcfebcb94f2a69ad0381be1146181ed"}
Sep 11 01:04:33.780371 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:33.780326    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-58fc6d6988-dvf8m" podStartSLOduration=1.947087131 podStartE2EDuration="4.780311657s" totalImagesPullingTime="2.833224526s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:04:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:04:33.778218962 +0000 UTC m=+389.175411134" watchObservedRunningTime="2026-09-11 01:04:33.780311657 +0000 UTC m=+389.177503866"
Sep 11 01:04:35.918119 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:35.918082    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 01:04:35.918399 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:35.918149    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 01:04:36.775547 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:36.775509    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p" event={"ID":"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5","Type":"ContainerStarted","Data":"4fbc66d49b319fc712544a24d851f7aae2cd0f124d94b48a222764d1834f79c6"}
Sep 11 01:04:36.775727 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:36.775684    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p"
Sep 11 01:04:36.777591 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:36.777559    2574 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-vwx5p container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 01:04:36.777726 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:36.777617    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p" podUID="d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 01:04:36.795510 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:36.795459    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p" podStartSLOduration=3.016948649 podStartE2EDuration="5.795443105s" totalImagesPullingTime="2.778494456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:04:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:04:36.793946257 +0000 UTC m=+392.191138430" watchObservedRunningTime="2026-09-11 01:04:36.795443105 +0000 UTC m=+392.192635283"
Sep 11 01:04:36.896344 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:36.896304    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:36.996164 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:36.996124    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:37.085380 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:37.085306    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:37.182274 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:37.182238    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:37.780579 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:37.780551    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p"
Sep 11 01:04:37.995075 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:37.995030    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:40.331883 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:40.331853    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:40.428026 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:40.427993    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:40.716944 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:40.716892    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:40.811311 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:40.811264    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:40.880272 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:40.880237    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:48.711198 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:48.711162    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5bdf494464-dk7ld"]
Sep 11 01:04:48.721444 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:48.721420    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5bdf494464-dk7ld"
Sep 11 01:04:48.725334 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:48.725300    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-5bnzz\""
Sep 11 01:04:48.725465 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:48.725334    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 11 01:04:48.725465 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:48.725383    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 11 01:04:48.725465 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:48.725452    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 11 01:04:48.733795 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:48.733775    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5bdf494464-dk7ld"]
Sep 11 01:04:48.767646 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:48.767613    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2e80d19c-5d88-4e0c-9a78-a45dfa9b3c7e-metrics-cert\") pod \"lws-controller-manager-5bdf494464-dk7ld\" (UID: \"2e80d19c-5d88-4e0c-9a78-a45dfa9b3c7e\") " pod="openshift-lws-operator/lws-controller-manager-5bdf494464-dk7ld"
Sep 11 01:04:48.767755 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:48.767654    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2e80d19c-5d88-4e0c-9a78-a45dfa9b3c7e-manager-config\") pod \"lws-controller-manager-5bdf494464-dk7ld\" (UID: \"2e80d19c-5d88-4e0c-9a78-a45dfa9b3c7e\") " pod="openshift-lws-operator/lws-controller-manager-5bdf494464-dk7ld"
Sep 11 01:04:48.767755 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:48.767709    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2e80d19c-5d88-4e0c-9a78-a45dfa9b3c7e-cert\") pod \"lws-controller-manager-5bdf494464-dk7ld\" (UID: \"2e80d19c-5d88-4e0c-9a78-a45dfa9b3c7e\") " pod="openshift-lws-operator/lws-controller-manager-5bdf494464-dk7ld"
Sep 11 01:04:48.767755 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:48.767736    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76gqr\" (UniqueName: \"kubernetes.io/projected/2e80d19c-5d88-4e0c-9a78-a45dfa9b3c7e-kube-api-access-76gqr\") pod \"lws-controller-manager-5bdf494464-dk7ld\" (UID: \"2e80d19c-5d88-4e0c-9a78-a45dfa9b3c7e\") " pod="openshift-lws-operator/lws-controller-manager-5bdf494464-dk7ld"
Sep 11 01:04:48.868557 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:48.868528    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2e80d19c-5d88-4e0c-9a78-a45dfa9b3c7e-metrics-cert\") pod \"lws-controller-manager-5bdf494464-dk7ld\" (UID: \"2e80d19c-5d88-4e0c-9a78-a45dfa9b3c7e\") " pod="openshift-lws-operator/lws-controller-manager-5bdf494464-dk7ld"
Sep 11 01:04:48.868557 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:48.868559    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2e80d19c-5d88-4e0c-9a78-a45dfa9b3c7e-manager-config\") pod \"lws-controller-manager-5bdf494464-dk7ld\" (UID: \"2e80d19c-5d88-4e0c-9a78-a45dfa9b3c7e\") " pod="openshift-lws-operator/lws-controller-manager-5bdf494464-dk7ld"
Sep 11 01:04:48.868764 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:48.868580    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2e80d19c-5d88-4e0c-9a78-a45dfa9b3c7e-cert\") pod \"lws-controller-manager-5bdf494464-dk7ld\" (UID: \"2e80d19c-5d88-4e0c-9a78-a45dfa9b3c7e\") " pod="openshift-lws-operator/lws-controller-manager-5bdf494464-dk7ld"
Sep 11 01:04:48.868764 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:48.868597    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76gqr\" (UniqueName: \"kubernetes.io/projected/2e80d19c-5d88-4e0c-9a78-a45dfa9b3c7e-kube-api-access-76gqr\") pod \"lws-controller-manager-5bdf494464-dk7ld\" (UID: \"2e80d19c-5d88-4e0c-9a78-a45dfa9b3c7e\") " pod="openshift-lws-operator/lws-controller-manager-5bdf494464-dk7ld"
Sep 11 01:04:48.869173 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:48.869154    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2e80d19c-5d88-4e0c-9a78-a45dfa9b3c7e-manager-config\") pod \"lws-controller-manager-5bdf494464-dk7ld\" (UID: \"2e80d19c-5d88-4e0c-9a78-a45dfa9b3c7e\") " pod="openshift-lws-operator/lws-controller-manager-5bdf494464-dk7ld"
Sep 11 01:04:48.871303 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:48.871278    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2e80d19c-5d88-4e0c-9a78-a45dfa9b3c7e-metrics-cert\") pod \"lws-controller-manager-5bdf494464-dk7ld\" (UID: \"2e80d19c-5d88-4e0c-9a78-a45dfa9b3c7e\") " pod="openshift-lws-operator/lws-controller-manager-5bdf494464-dk7ld"
Sep 11 01:04:48.871384 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:48.871335    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2e80d19c-5d88-4e0c-9a78-a45dfa9b3c7e-cert\") pod \"lws-controller-manager-5bdf494464-dk7ld\" (UID: \"2e80d19c-5d88-4e0c-9a78-a45dfa9b3c7e\") " pod="openshift-lws-operator/lws-controller-manager-5bdf494464-dk7ld"
Sep 11 01:04:48.876857 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:48.876828    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76gqr\" (UniqueName: \"kubernetes.io/projected/2e80d19c-5d88-4e0c-9a78-a45dfa9b3c7e-kube-api-access-76gqr\") pod \"lws-controller-manager-5bdf494464-dk7ld\" (UID: \"2e80d19c-5d88-4e0c-9a78-a45dfa9b3c7e\") " pod="openshift-lws-operator/lws-controller-manager-5bdf494464-dk7ld"
Sep 11 01:04:49.030901 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:49.030836    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5bdf494464-dk7ld"
Sep 11 01:04:49.321485 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:49.321404    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5bdf494464-dk7ld"]
Sep 11 01:04:49.324645 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:04:49.324619    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e80d19c_5d88_4e0c_9a78_a45dfa9b3c7e.slice/crio-dc2ec66d05b67058318f968773caacb218558957914264d83ae5f4f9cebfad26 WatchSource:0}: Error finding container dc2ec66d05b67058318f968773caacb218558957914264d83ae5f4f9cebfad26: Status 404 returned error can't find the container with id dc2ec66d05b67058318f968773caacb218558957914264d83ae5f4f9cebfad26
Sep 11 01:04:49.830001 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:49.829953    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5bdf494464-dk7ld" event={"ID":"2e80d19c-5d88-4e0c-9a78-a45dfa9b3c7e","Type":"ContainerStarted","Data":"dc2ec66d05b67058318f968773caacb218558957914264d83ae5f4f9cebfad26"}
Sep 11 01:04:51.009593 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:51.009559    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:51.170948 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:51.170882    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:51.333140 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:51.333109    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:51.423184 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:51.423105    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:51.839835 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:51.839751    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5bdf494464-dk7ld" event={"ID":"2e80d19c-5d88-4e0c-9a78-a45dfa9b3c7e","Type":"ContainerStarted","Data":"154492b7425b659c602819f990e61871356e067d9535023349cab154b911bbcd"}
Sep 11 01:04:51.839999 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:51.839892    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5bdf494464-dk7ld"
Sep 11 01:04:51.875869 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:51.875822    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5bdf494464-dk7ld" podStartSLOduration=2.383819259 podStartE2EDuration="3.875808894s" totalImagesPullingTime="1.491989635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:04:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:04:51.874250005 +0000 UTC m=+407.271442178" watchObservedRunningTime="2026-09-11 01:04:51.875808894 +0000 UTC m=+407.273001066"
Sep 11 01:04:52.218860 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.218818    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p"]
Sep 11 01:04:52.219334 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.219091    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p" podUID="d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5" containerName="discovery" containerID="cri-o://4fbc66d49b319fc712544a24d851f7aae2cd0f124d94b48a222764d1834f79c6" gracePeriod=30
Sep 11 01:04:52.398416 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.397700    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:52.460029 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.460006    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p"
Sep 11 01:04:52.499365 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.499286    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-istio-token\") pod \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\" (UID: \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\") "
Sep 11 01:04:52.499365 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.499325    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-istio-csr-dns-cert\") pod \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\" (UID: \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\") "
Sep 11 01:04:52.499365 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.499349    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-istio-csr-ca-configmap\") pod \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\" (UID: \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\") "
Sep 11 01:04:52.499618 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.499394    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-cacerts\") pod \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\" (UID: \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\") "
Sep 11 01:04:52.499618 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.499429    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-istio-kubeconfig\") pod \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\" (UID: \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\") "
Sep 11 01:04:52.499618 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.499453    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-local-certs\") pod \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\" (UID: \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\") "
Sep 11 01:04:52.499618 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.499483    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjrtp\" (UniqueName: \"kubernetes.io/projected/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-kube-api-access-bjrtp\") pod \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\" (UID: \"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5\") "
Sep 11 01:04:52.499823 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.499790    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5" (UID: "d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:04:52.501743 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.501676    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:04:52.502314 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.502288    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5" (UID: "d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:04:52.502775 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.502741    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-kube-api-access-bjrtp" (OuterVolumeSpecName: "kube-api-access-bjrtp") pod "d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5" (UID: "d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5"). InnerVolumeSpecName "kube-api-access-bjrtp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:04:52.502864 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.502780    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-cacerts" (OuterVolumeSpecName: "cacerts") pod "d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5" (UID: "d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:04:52.502864 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.502791    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-istio-token" (OuterVolumeSpecName: "istio-token") pod "d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5" (UID: "d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:04:52.502864 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.502845    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-local-certs" (OuterVolumeSpecName: "local-certs") pod "d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5" (UID: "d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:04:52.503594 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.503572    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5" (UID: "d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:04:52.600681 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.600646    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-istio-token\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:04:52.600681 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.600675    2574 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-istio-csr-dns-cert\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:04:52.600681 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.600685    2574 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-istio-csr-ca-configmap\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:04:52.600896 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.600693    2574 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-cacerts\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:04:52.600896 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.600703    2574 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-istio-kubeconfig\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:04:52.600896 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.600711    2574 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-local-certs\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:04:52.600896 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.600720    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjrtp\" (UniqueName: \"kubernetes.io/projected/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5-kube-api-access-bjrtp\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:04:52.852882 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.852789    2574 generic.go:356] "Generic (PLEG): container finished" podID="d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5" containerID="4fbc66d49b319fc712544a24d851f7aae2cd0f124d94b48a222764d1834f79c6" exitCode=0
Sep 11 01:04:52.852882 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.852836    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p"
Sep 11 01:04:52.852882 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.852853    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p" event={"ID":"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5","Type":"ContainerDied","Data":"4fbc66d49b319fc712544a24d851f7aae2cd0f124d94b48a222764d1834f79c6"}
Sep 11 01:04:52.853146 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.852937    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p" event={"ID":"d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5","Type":"ContainerDied","Data":"03063b10150e48355f698afe08c2b9713bcfebcb94f2a69ad0381be1146181ed"}
Sep 11 01:04:52.853146 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.852968    2574 scope.go:117] "RemoveContainer" containerID="4fbc66d49b319fc712544a24d851f7aae2cd0f124d94b48a222764d1834f79c6"
Sep 11 01:04:52.862185 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.862162    2574 scope.go:117] "RemoveContainer" containerID="4fbc66d49b319fc712544a24d851f7aae2cd0f124d94b48a222764d1834f79c6"
Sep 11 01:04:52.862491 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:04:52.862465    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fbc66d49b319fc712544a24d851f7aae2cd0f124d94b48a222764d1834f79c6\": container with ID starting with 4fbc66d49b319fc712544a24d851f7aae2cd0f124d94b48a222764d1834f79c6 not found: ID does not exist" containerID="4fbc66d49b319fc712544a24d851f7aae2cd0f124d94b48a222764d1834f79c6"
Sep 11 01:04:52.862567 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.862496    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fbc66d49b319fc712544a24d851f7aae2cd0f124d94b48a222764d1834f79c6"} err="failed to get container status \"4fbc66d49b319fc712544a24d851f7aae2cd0f124d94b48a222764d1834f79c6\": rpc error: code = NotFound desc = could not find container \"4fbc66d49b319fc712544a24d851f7aae2cd0f124d94b48a222764d1834f79c6\": container with ID starting with 4fbc66d49b319fc712544a24d851f7aae2cd0f124d94b48a222764d1834f79c6 not found: ID does not exist"
Sep 11 01:04:52.886751 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.886711    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p"]
Sep 11 01:04:52.898749 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:52.898712    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vwx5p"]
Sep 11 01:04:53.099245 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:04:53.099214    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5" path="/var/lib/kubelet/pods/d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5/volumes"
Sep 11 01:05:02.855745 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:05:02.855710    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5bdf494464-dk7ld"
Sep 11 01:05:24.815895 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:05:24.815859    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:05:24.916154 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:05:24.916127    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:05:26.427370 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:05:26.427288    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:05:26.526611 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:05:26.526580    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:05:28.166622 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:05:28.166586    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:05:28.237567 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:05:28.237537    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:05:41.469888 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:05:41.469855    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:05:41.564545 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:05:41.564505    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:05:48.472071 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:05:48.472039    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:05:48.528512 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:05:48.528490    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:05:58.608472 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:05:58.608434    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:00.994904 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:00.994867    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:01.390340 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:01.390223    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:03.467671 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:03.467637    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:03.563275 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:03.563235    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:03.659895 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:03.659861    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:04.197796 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:04.197766    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:04.597819 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:04.597734    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:05.643485 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:05.643446    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:06.643209 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:06.643171    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:06.744041 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:06.744013    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:08.025167 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:08.025131    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:08.648042 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:08.648007    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:09.706188 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:09.706151    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ws2mz"]
Sep 11 01:06:09.706566 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:09.706446    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5" containerName="discovery"
Sep 11 01:06:09.706566 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:09.706458    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5" containerName="discovery"
Sep 11 01:06:09.706566 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:09.706511    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8ca41d9-7ec4-4344-a9d5-b7d10fdb34e5" containerName="discovery"
Sep 11 01:06:09.709448 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:09.709432    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ws2mz"
Sep 11 01:06:09.711790 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:09.711774    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-wpkxp\""
Sep 11 01:06:09.711887 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:09.711872    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 11 01:06:09.712313 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:09.712295    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 11 01:06:09.712653 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:09.712641    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 11 01:06:09.719300 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:09.719280    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ws2mz"]
Sep 11 01:06:09.816407 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:09.816375    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmlcv\" (UniqueName: \"kubernetes.io/projected/cb5c9843-cc67-456f-a22d-fa23447c7f4b-kube-api-access-zmlcv\") pod \"dns-operator-controller-manager-7b84cd7c9c-ws2mz\" (UID: \"cb5c9843-cc67-456f-a22d-fa23447c7f4b\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ws2mz"
Sep 11 01:06:09.917596 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:09.917563    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmlcv\" (UniqueName: \"kubernetes.io/projected/cb5c9843-cc67-456f-a22d-fa23447c7f4b-kube-api-access-zmlcv\") pod \"dns-operator-controller-manager-7b84cd7c9c-ws2mz\" (UID: \"cb5c9843-cc67-456f-a22d-fa23447c7f4b\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ws2mz"
Sep 11 01:06:09.925673 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:09.925644    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmlcv\" (UniqueName: \"kubernetes.io/projected/cb5c9843-cc67-456f-a22d-fa23447c7f4b-kube-api-access-zmlcv\") pod \"dns-operator-controller-manager-7b84cd7c9c-ws2mz\" (UID: \"cb5c9843-cc67-456f-a22d-fa23447c7f4b\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ws2mz"
Sep 11 01:06:10.019719 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:10.019659    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ws2mz"
Sep 11 01:06:10.289080 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:10.289011    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ws2mz"]
Sep 11 01:06:10.292002 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:06:10.291966    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb5c9843_cc67_456f_a22d_fa23447c7f4b.slice/crio-20105b052391aee485ad457d78d4ed881e5c79cf965421d742a3ba02a39b4d56 WatchSource:0}: Error finding container 20105b052391aee485ad457d78d4ed881e5c79cf965421d742a3ba02a39b4d56: Status 404 returned error can't find the container with id 20105b052391aee485ad457d78d4ed881e5c79cf965421d742a3ba02a39b4d56
Sep 11 01:06:11.002981 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:11.002901    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:11.107165 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:11.107115    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:11.166741 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:11.166703    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ws2mz" event={"ID":"cb5c9843-cc67-456f-a22d-fa23447c7f4b","Type":"ContainerStarted","Data":"20105b052391aee485ad457d78d4ed881e5c79cf965421d742a3ba02a39b4d56"}
Sep 11 01:06:13.176873 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:13.176844    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ws2mz" event={"ID":"cb5c9843-cc67-456f-a22d-fa23447c7f4b","Type":"ContainerStarted","Data":"c6a68942b0d1edaf36625b19a85939cc140f9bef21d7e0fc8947a36b50662877"}
Sep 11 01:06:13.177351 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:13.177135    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ws2mz"
Sep 11 01:06:13.195028 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:13.194980    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ws2mz" podStartSLOduration=1.645976911 podStartE2EDuration="4.194967256s" totalImagesPullingTime="2.548990345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:06:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:06:13.19357013 +0000 UTC m=+488.590762304" watchObservedRunningTime="2026-09-11 01:06:13.194967256 +0000 UTC m=+488.592159429"
Sep 11 01:06:14.381390 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:14.381350    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-jmkt5"]
Sep 11 01:06:14.381829 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:14.381630    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:14.384953 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:14.384937    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-jmkt5"
Sep 11 01:06:14.387401 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:14.387375    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 11 01:06:14.387526 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:14.387402    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7d58x\""
Sep 11 01:06:14.387526 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:14.387375    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 11 01:06:14.391633 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:14.391609    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-jmkt5"]
Sep 11 01:06:14.457101 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:14.457075    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6zbl\" (UniqueName: \"kubernetes.io/projected/42094776-f11a-4606-a02a-8ab6829214bf-kube-api-access-l6zbl\") pod \"kuadrant-console-plugin-fbdd4f445-jmkt5\" (UID: \"42094776-f11a-4606-a02a-8ab6829214bf\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-jmkt5"
Sep 11 01:06:14.457228 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:14.457119    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/42094776-f11a-4606-a02a-8ab6829214bf-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-jmkt5\" (UID: \"42094776-f11a-4606-a02a-8ab6829214bf\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-jmkt5"
Sep 11 01:06:14.457228 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:14.457144    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/42094776-f11a-4606-a02a-8ab6829214bf-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-jmkt5\" (UID: \"42094776-f11a-4606-a02a-8ab6829214bf\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-jmkt5"
Sep 11 01:06:14.505552 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:14.505525    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:14.558269 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:14.558238    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/42094776-f11a-4606-a02a-8ab6829214bf-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-jmkt5\" (UID: \"42094776-f11a-4606-a02a-8ab6829214bf\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-jmkt5"
Sep 11 01:06:14.558399 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:14.558325    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6zbl\" (UniqueName: \"kubernetes.io/projected/42094776-f11a-4606-a02a-8ab6829214bf-kube-api-access-l6zbl\") pod \"kuadrant-console-plugin-fbdd4f445-jmkt5\" (UID: \"42094776-f11a-4606-a02a-8ab6829214bf\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-jmkt5"
Sep 11 01:06:14.558399 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:14.558380    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/42094776-f11a-4606-a02a-8ab6829214bf-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-jmkt5\" (UID: \"42094776-f11a-4606-a02a-8ab6829214bf\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-jmkt5"
Sep 11 01:06:14.558981 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:14.558961    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/42094776-f11a-4606-a02a-8ab6829214bf-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-jmkt5\" (UID: \"42094776-f11a-4606-a02a-8ab6829214bf\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-jmkt5"
Sep 11 01:06:14.560775 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:14.560747    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/42094776-f11a-4606-a02a-8ab6829214bf-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-jmkt5\" (UID: \"42094776-f11a-4606-a02a-8ab6829214bf\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-jmkt5"
Sep 11 01:06:14.566604 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:14.566583    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6zbl\" (UniqueName: \"kubernetes.io/projected/42094776-f11a-4606-a02a-8ab6829214bf-kube-api-access-l6zbl\") pod \"kuadrant-console-plugin-fbdd4f445-jmkt5\" (UID: \"42094776-f11a-4606-a02a-8ab6829214bf\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-jmkt5"
Sep 11 01:06:14.694204 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:14.694185    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-jmkt5"
Sep 11 01:06:14.961978 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:14.961887    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-jmkt5"]
Sep 11 01:06:14.964899 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:06:14.964870    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod42094776_f11a_4606_a02a_8ab6829214bf.slice/crio-27b4c3d2b68f64d3bca6bc3ec541b8225b396d18ac7b6935f9b4712efda31278 WatchSource:0}: Error finding container 27b4c3d2b68f64d3bca6bc3ec541b8225b396d18ac7b6935f9b4712efda31278: Status 404 returned error can't find the container with id 27b4c3d2b68f64d3bca6bc3ec541b8225b396d18ac7b6935f9b4712efda31278
Sep 11 01:06:15.002457 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:15.002420    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:15.062492 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:15.062464    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:15.192033 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:15.192001    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-jmkt5" event={"ID":"42094776-f11a-4606-a02a-8ab6829214bf","Type":"ContainerStarted","Data":"27b4c3d2b68f64d3bca6bc3ec541b8225b396d18ac7b6935f9b4712efda31278"}
Sep 11 01:06:15.396657 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:15.396570    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:15.452516 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:15.452490    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:20.219177 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:20.219141    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-jmkt5" event={"ID":"42094776-f11a-4606-a02a-8ab6829214bf","Type":"ContainerStarted","Data":"179311c198a7d4bcdd00553a229299bce96067251f704bffb775e29f020ae5ac"}
Sep 11 01:06:20.233521 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:20.233477    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-jmkt5" podStartSLOduration=1.57145219 podStartE2EDuration="6.233459991s" totalImagesPullingTime="4.662007801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:06:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:06:20.233018622 +0000 UTC m=+495.630210796" watchObservedRunningTime="2026-09-11 01:06:20.233459991 +0000 UTC m=+495.630652169"
Sep 11 01:06:20.416803 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:20.416750    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:20.514276 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:20.514193    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:20.568270 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:20.568237    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:24.183217 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:24.183184    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ws2mz"
Sep 11 01:06:25.674479 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:25.674446    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-rdhcs"]
Sep 11 01:06:25.678184 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:25.678159    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rdhcs"
Sep 11 01:06:25.682347 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:25.682324    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-gkmpg\""
Sep 11 01:06:25.689228 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:25.689203    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-rdhcs"]
Sep 11 01:06:25.859544 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:25.859506    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5bgb\" (UniqueName: \"kubernetes.io/projected/954c463d-8df1-4d71-a781-ddbb9f1c1282-kube-api-access-m5bgb\") pod \"limitador-operator-controller-manager-74495c69f8-rdhcs\" (UID: \"954c463d-8df1-4d71-a781-ddbb9f1c1282\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rdhcs"
Sep 11 01:06:25.960345 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:25.960270    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5bgb\" (UniqueName: \"kubernetes.io/projected/954c463d-8df1-4d71-a781-ddbb9f1c1282-kube-api-access-m5bgb\") pod \"limitador-operator-controller-manager-74495c69f8-rdhcs\" (UID: \"954c463d-8df1-4d71-a781-ddbb9f1c1282\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rdhcs"
Sep 11 01:06:25.968400 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:25.968370    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5bgb\" (UniqueName: \"kubernetes.io/projected/954c463d-8df1-4d71-a781-ddbb9f1c1282-kube-api-access-m5bgb\") pod \"limitador-operator-controller-manager-74495c69f8-rdhcs\" (UID: \"954c463d-8df1-4d71-a781-ddbb9f1c1282\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rdhcs"
Sep 11 01:06:25.988946 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:25.988903    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rdhcs"
Sep 11 01:06:26.278480 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:26.278415    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-rdhcs"]
Sep 11 01:06:26.280881 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:06:26.280857    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod954c463d_8df1_4d71_a781_ddbb9f1c1282.slice/crio-7ce9f0649fef40633972c767aeb61216c5d957f40aebe1a5705ea6e7851d8e12 WatchSource:0}: Error finding container 7ce9f0649fef40633972c767aeb61216c5d957f40aebe1a5705ea6e7851d8e12: Status 404 returned error can't find the container with id 7ce9f0649fef40633972c767aeb61216c5d957f40aebe1a5705ea6e7851d8e12
Sep 11 01:06:26.975992 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:26.975956    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:27.074014 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:27.073979    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:27.255201 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:27.255124    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rdhcs" event={"ID":"954c463d-8df1-4d71-a781-ddbb9f1c1282","Type":"ContainerStarted","Data":"7ce9f0649fef40633972c767aeb61216c5d957f40aebe1a5705ea6e7851d8e12"}
Sep 11 01:06:28.260421 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:28.260389    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rdhcs" event={"ID":"954c463d-8df1-4d71-a781-ddbb9f1c1282","Type":"ContainerStarted","Data":"be32cd5d2c9b7d6fa2da592d74b0e6c2350107b5a9937bf0607862264ee21fbe"}
Sep 11 01:06:28.260823 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:28.260459    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rdhcs"
Sep 11 01:06:28.277125 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:28.277080    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rdhcs" podStartSLOduration=1.738772143 podStartE2EDuration="3.277065614s" totalImagesPullingTime="1.538293471s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:06:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:06:28.275595923 +0000 UTC m=+503.672788118" watchObservedRunningTime="2026-09-11 01:06:28.277065614 +0000 UTC m=+503.674257787"
Sep 11 01:06:31.432683 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:31.432647    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:31.516056 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:31.516024    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:35.092253 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:35.092216    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:35.153444 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:35.153414    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:35.284831 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:35.284790    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:35.343037 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:35.342955    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:39.266852 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:39.266825    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rdhcs"
Sep 11 01:06:40.409364 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:40.409330    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:40.737479 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:40.737366    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:40.848528 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:40.848493    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:40.946475 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:40.946440    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:41.051521 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:41.051458    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:41.166569 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:41.166515    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:41.252571 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:41.252543    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:47.058473 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:47.058436    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:47.158958 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:47.158889    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:53.978072 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:53.978013    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:54.226520 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:54.226486    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:58.171006 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:58.170892    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:06:58.233687 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:06:58.233655    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:07:08.877375 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:08.877340    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:07:08.989117 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:08.989089    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:07:09.090207 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:09.090174    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:07:09.741126 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:09.741092    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:07:09.823445 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:09.823401    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:07:09.823717 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:09.823696    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-qvsvq"]
Sep 11 01:07:09.827030 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:09.827016    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-qvsvq"
Sep 11 01:07:09.829447 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:09.829427    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rg9lb\""
Sep 11 01:07:09.836455 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:09.836428    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-qvsvq"]
Sep 11 01:07:09.927957 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:09.927900    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdw6b\" (UniqueName: \"kubernetes.io/projected/f1d2c207-1d19-42cc-b33f-b930c63fa5fe-kube-api-access-pdw6b\") pod \"authorino-59c974bd48-qvsvq\" (UID: \"f1d2c207-1d19-42cc-b33f-b930c63fa5fe\") " pod="kuadrant-system/authorino-59c974bd48-qvsvq"
Sep 11 01:07:10.029306 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:10.029220    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdw6b\" (UniqueName: \"kubernetes.io/projected/f1d2c207-1d19-42cc-b33f-b930c63fa5fe-kube-api-access-pdw6b\") pod \"authorino-59c974bd48-qvsvq\" (UID: \"f1d2c207-1d19-42cc-b33f-b930c63fa5fe\") " pod="kuadrant-system/authorino-59c974bd48-qvsvq"
Sep 11 01:07:10.040778 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:10.040753    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdw6b\" (UniqueName: \"kubernetes.io/projected/f1d2c207-1d19-42cc-b33f-b930c63fa5fe-kube-api-access-pdw6b\") pod \"authorino-59c974bd48-qvsvq\" (UID: \"f1d2c207-1d19-42cc-b33f-b930c63fa5fe\") " pod="kuadrant-system/authorino-59c974bd48-qvsvq"
Sep 11 01:07:10.136121 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:10.136064    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-qvsvq"
Sep 11 01:07:10.411977 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:10.411945    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-qvsvq"]
Sep 11 01:07:10.415159 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:07:10.415130    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf1d2c207_1d19_42cc_b33f_b930c63fa5fe.slice/crio-0619995c0f687270f9f314095c211e0bd7e5ed37f2d2497a1a1425d111c67cd7 WatchSource:0}: Error finding container 0619995c0f687270f9f314095c211e0bd7e5ed37f2d2497a1a1425d111c67cd7: Status 404 returned error can't find the container with id 0619995c0f687270f9f314095c211e0bd7e5ed37f2d2497a1a1425d111c67cd7
Sep 11 01:07:10.440952 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:10.440894    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-qvsvq" event={"ID":"f1d2c207-1d19-42cc-b33f-b930c63fa5fe","Type":"ContainerStarted","Data":"0619995c0f687270f9f314095c211e0bd7e5ed37f2d2497a1a1425d111c67cd7"}
Sep 11 01:07:13.456134 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:13.456094    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-qvsvq" event={"ID":"f1d2c207-1d19-42cc-b33f-b930c63fa5fe","Type":"ContainerStarted","Data":"58761968c51b6b1bc6aa72d764c2c7184d72f21f00ff4511fb45d798967596c5"}
Sep 11 01:07:13.469804 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:13.469758    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-qvsvq" podStartSLOduration=2.124606893 podStartE2EDuration="4.469744508s" totalImagesPullingTime="2.345137615s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:07:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:07:13.469040269 +0000 UTC m=+548.866232441" watchObservedRunningTime="2026-09-11 01:07:13.469744508 +0000 UTC m=+548.866936681"
Sep 11 01:07:14.142109 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:14.142076    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:07:14.193330 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:14.193298    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:07:17.916356 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:17.916320    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:07:22.353008 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:22.352971    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:07:25.153568 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:25.153536    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:07:25.254468 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:25.254428    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:07:27.509734 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:27.509706    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:07:27.606611 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:27.606575    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:07:28.020583 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:28.020542    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:07:40.430000 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:40.429956    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:07:40.603001 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:40.602959    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:07:42.880351 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:42.880310    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-qvsvq"]
Sep 11 01:07:42.880778 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:42.880511    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-qvsvq" podUID="f1d2c207-1d19-42cc-b33f-b930c63fa5fe" containerName="authorino" containerID="cri-o://58761968c51b6b1bc6aa72d764c2c7184d72f21f00ff4511fb45d798967596c5" gracePeriod=30
Sep 11 01:07:43.120895 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:43.120872    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-qvsvq"
Sep 11 01:07:43.205613 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:43.205584    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pdw6b\" (UniqueName: \"kubernetes.io/projected/f1d2c207-1d19-42cc-b33f-b930c63fa5fe-kube-api-access-pdw6b\") pod \"f1d2c207-1d19-42cc-b33f-b930c63fa5fe\" (UID: \"f1d2c207-1d19-42cc-b33f-b930c63fa5fe\") "
Sep 11 01:07:43.207989 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:43.207965    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1d2c207-1d19-42cc-b33f-b930c63fa5fe-kube-api-access-pdw6b" (OuterVolumeSpecName: "kube-api-access-pdw6b") pod "f1d2c207-1d19-42cc-b33f-b930c63fa5fe" (UID: "f1d2c207-1d19-42cc-b33f-b930c63fa5fe"). InnerVolumeSpecName "kube-api-access-pdw6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:07:43.307483 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:43.307440    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pdw6b\" (UniqueName: \"kubernetes.io/projected/f1d2c207-1d19-42cc-b33f-b930c63fa5fe-kube-api-access-pdw6b\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:07:43.583454 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:43.583412    2574 generic.go:356] "Generic (PLEG): container finished" podID="f1d2c207-1d19-42cc-b33f-b930c63fa5fe" containerID="58761968c51b6b1bc6aa72d764c2c7184d72f21f00ff4511fb45d798967596c5" exitCode=0
Sep 11 01:07:43.583454 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:43.583451    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-qvsvq"
Sep 11 01:07:43.583648 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:43.583465    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-qvsvq" event={"ID":"f1d2c207-1d19-42cc-b33f-b930c63fa5fe","Type":"ContainerDied","Data":"58761968c51b6b1bc6aa72d764c2c7184d72f21f00ff4511fb45d798967596c5"}
Sep 11 01:07:43.583648 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:43.583504    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-qvsvq" event={"ID":"f1d2c207-1d19-42cc-b33f-b930c63fa5fe","Type":"ContainerDied","Data":"0619995c0f687270f9f314095c211e0bd7e5ed37f2d2497a1a1425d111c67cd7"}
Sep 11 01:07:43.583648 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:43.583524    2574 scope.go:117] "RemoveContainer" containerID="58761968c51b6b1bc6aa72d764c2c7184d72f21f00ff4511fb45d798967596c5"
Sep 11 01:07:43.592079 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:43.592062    2574 scope.go:117] "RemoveContainer" containerID="58761968c51b6b1bc6aa72d764c2c7184d72f21f00ff4511fb45d798967596c5"
Sep 11 01:07:43.592313 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:07:43.592296    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58761968c51b6b1bc6aa72d764c2c7184d72f21f00ff4511fb45d798967596c5\": container with ID starting with 58761968c51b6b1bc6aa72d764c2c7184d72f21f00ff4511fb45d798967596c5 not found: ID does not exist" containerID="58761968c51b6b1bc6aa72d764c2c7184d72f21f00ff4511fb45d798967596c5"
Sep 11 01:07:43.592357 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:43.592319    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58761968c51b6b1bc6aa72d764c2c7184d72f21f00ff4511fb45d798967596c5"} err="failed to get container status \"58761968c51b6b1bc6aa72d764c2c7184d72f21f00ff4511fb45d798967596c5\": rpc error: code = NotFound desc = could not find container \"58761968c51b6b1bc6aa72d764c2c7184d72f21f00ff4511fb45d798967596c5\": container with ID starting with 58761968c51b6b1bc6aa72d764c2c7184d72f21f00ff4511fb45d798967596c5 not found: ID does not exist"
Sep 11 01:07:43.602825 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:43.602799    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-qvsvq"]
Sep 11 01:07:43.604074 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:43.604046    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-qvsvq"]
Sep 11 01:07:45.099255 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:45.099221    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1d2c207-1d19-42cc-b33f-b930c63fa5fe" path="/var/lib/kubelet/pods/f1d2c207-1d19-42cc-b33f-b930c63fa5fe/volumes"
Sep 11 01:07:47.840560 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:47.840525    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:07:47.920196 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:47.920162    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:07:47.977293 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:47.977232    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:07:48.106796 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:48.106692    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:07:48.262112 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:48.262079    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:07:51.332305 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:51.332272    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:07:51.431146 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:07:51.431102    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:08:23.830669 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:23.830633    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:08:25.016590 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:25.016550    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:08:25.313137 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:25.313059    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:08:25.413869 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:25.413841    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:08:25.571507 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:25.571432    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:08:25.847978 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:25.847895    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:08:25.957303 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:25.957277    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:08:25.965070 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:25.965045    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-dxdkg"]
Sep 11 01:08:25.965460 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:25.965440    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1d2c207-1d19-42cc-b33f-b930c63fa5fe" containerName="authorino"
Sep 11 01:08:25.965460 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:25.965461    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1d2c207-1d19-42cc-b33f-b930c63fa5fe" containerName="authorino"
Sep 11 01:08:25.965603 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:25.965545    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1d2c207-1d19-42cc-b33f-b930c63fa5fe" containerName="authorino"
Sep 11 01:08:25.969596 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:25.969579    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-dxdkg"
Sep 11 01:08:25.971592 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:25.971562    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-rh479\""
Sep 11 01:08:25.971703 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:25.971679    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 11 01:08:25.977495 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:25.977469    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-dxdkg"]
Sep 11 01:08:26.040381 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:26.040339    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef51a3ee-99ca-4c0f-bf70-1bf2b8791cc5-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-dxdkg\" (UID: \"ef51a3ee-99ca-4c0f-bf70-1bf2b8791cc5\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-dxdkg"
Sep 11 01:08:26.040381 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:26.040387    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6trq\" (UniqueName: \"kubernetes.io/projected/ef51a3ee-99ca-4c0f-bf70-1bf2b8791cc5-kube-api-access-s6trq\") pod \"payload-pre-processing-59fbc47bf8-dxdkg\" (UID: \"ef51a3ee-99ca-4c0f-bf70-1bf2b8791cc5\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-dxdkg"
Sep 11 01:08:26.141277 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:26.141209    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef51a3ee-99ca-4c0f-bf70-1bf2b8791cc5-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-dxdkg\" (UID: \"ef51a3ee-99ca-4c0f-bf70-1bf2b8791cc5\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-dxdkg"
Sep 11 01:08:26.141277 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:26.141252    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6trq\" (UniqueName: \"kubernetes.io/projected/ef51a3ee-99ca-4c0f-bf70-1bf2b8791cc5-kube-api-access-s6trq\") pod \"payload-pre-processing-59fbc47bf8-dxdkg\" (UID: \"ef51a3ee-99ca-4c0f-bf70-1bf2b8791cc5\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-dxdkg"
Sep 11 01:08:26.141895 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:26.141877    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef51a3ee-99ca-4c0f-bf70-1bf2b8791cc5-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-dxdkg\" (UID: \"ef51a3ee-99ca-4c0f-bf70-1bf2b8791cc5\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-dxdkg"
Sep 11 01:08:26.152194 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:26.152176    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6trq\" (UniqueName: \"kubernetes.io/projected/ef51a3ee-99ca-4c0f-bf70-1bf2b8791cc5-kube-api-access-s6trq\") pod \"payload-pre-processing-59fbc47bf8-dxdkg\" (UID: \"ef51a3ee-99ca-4c0f-bf70-1bf2b8791cc5\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-dxdkg"
Sep 11 01:08:26.280126 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:26.280098    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-dxdkg"
Sep 11 01:08:26.551153 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:26.551123    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-dxdkg"]
Sep 11 01:08:26.554132 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:08:26.554108    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef51a3ee_99ca_4c0f_bf70_1bf2b8791cc5.slice/crio-2e20612e6ba97ab40da5fc0fb155ee04eef2f5e2187447e7dfbadd0aedc520f1 WatchSource:0}: Error finding container 2e20612e6ba97ab40da5fc0fb155ee04eef2f5e2187447e7dfbadd0aedc520f1: Status 404 returned error can't find the container with id 2e20612e6ba97ab40da5fc0fb155ee04eef2f5e2187447e7dfbadd0aedc520f1
Sep 11 01:08:26.762091 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:26.762055    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-dxdkg" event={"ID":"ef51a3ee-99ca-4c0f-bf70-1bf2b8791cc5","Type":"ContainerStarted","Data":"2e20612e6ba97ab40da5fc0fb155ee04eef2f5e2187447e7dfbadd0aedc520f1"}
Sep 11 01:08:29.777561 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:29.777527    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-dxdkg" event={"ID":"ef51a3ee-99ca-4c0f-bf70-1bf2b8791cc5","Type":"ContainerStarted","Data":"febbe5511f11a265c52e7fbf0d2aa018b1f0300f22ac4a2f00a8ff8eee35be2f"}
Sep 11 01:08:29.778016 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:29.777598    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-dxdkg"
Sep 11 01:08:29.793533 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:29.793488    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-dxdkg" podStartSLOduration=2.637083958 podStartE2EDuration="4.793469898s" totalImagesPullingTime="2.15638594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:08:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:08:29.792625422 +0000 UTC m=+625.189817594" watchObservedRunningTime="2026-09-11 01:08:29.793469898 +0000 UTC m=+625.190662071"
Sep 11 01:08:36.477573 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:36.477522    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:08:36.573671 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:36.573624    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:08:38.770896 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:38.770861    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:08:40.784469 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:40.784443    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-dxdkg"
Sep 11 01:08:47.717950 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:47.717898    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:08:53.279297 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:53.279264    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-gk5b8"]
Sep 11 01:08:53.282748 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:53.282728    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gk5b8"
Sep 11 01:08:53.285115 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:53.285093    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 11 01:08:53.285978 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:53.285959    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 11 01:08:53.285978 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:53.285971    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 11 01:08:53.286149 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:53.286096    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6tkxm\""
Sep 11 01:08:53.292080 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:53.292054    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-gk5b8"]
Sep 11 01:08:53.310323 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:53.310293    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:08:53.359120 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:53.359094    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb818e57-a9c5-41ac-b057-df2fb48e5b86-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gk5b8\" (UID: \"eb818e57-a9c5-41ac-b057-df2fb48e5b86\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gk5b8"
Sep 11 01:08:53.359293 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:53.359128    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb818e57-a9c5-41ac-b057-df2fb48e5b86-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gk5b8\" (UID: \"eb818e57-a9c5-41ac-b057-df2fb48e5b86\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gk5b8"
Sep 11 01:08:53.359293 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:53.359164    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb818e57-a9c5-41ac-b057-df2fb48e5b86-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gk5b8\" (UID: \"eb818e57-a9c5-41ac-b057-df2fb48e5b86\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gk5b8"
Sep 11 01:08:53.359293 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:53.359239    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb818e57-a9c5-41ac-b057-df2fb48e5b86-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gk5b8\" (UID: \"eb818e57-a9c5-41ac-b057-df2fb48e5b86\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gk5b8"
Sep 11 01:08:53.359293 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:53.359279    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpz8p\" (UniqueName: \"kubernetes.io/projected/eb818e57-a9c5-41ac-b057-df2fb48e5b86-kube-api-access-dpz8p\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gk5b8\" (UID: \"eb818e57-a9c5-41ac-b057-df2fb48e5b86\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gk5b8"
Sep 11 01:08:53.359481 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:53.359343    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb818e57-a9c5-41ac-b057-df2fb48e5b86-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gk5b8\" (UID: \"eb818e57-a9c5-41ac-b057-df2fb48e5b86\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gk5b8"
Sep 11 01:08:53.460032 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:53.459994    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb818e57-a9c5-41ac-b057-df2fb48e5b86-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gk5b8\" (UID: \"eb818e57-a9c5-41ac-b057-df2fb48e5b86\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gk5b8"
Sep 11 01:08:53.460032 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:53.460030    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb818e57-a9c5-41ac-b057-df2fb48e5b86-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gk5b8\" (UID: \"eb818e57-a9c5-41ac-b057-df2fb48e5b86\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gk5b8"
Sep 11 01:08:53.460271 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:53.460046    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpz8p\" (UniqueName: \"kubernetes.io/projected/eb818e57-a9c5-41ac-b057-df2fb48e5b86-kube-api-access-dpz8p\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gk5b8\" (UID: \"eb818e57-a9c5-41ac-b057-df2fb48e5b86\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gk5b8"
Sep 11 01:08:53.460271 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:53.460078    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb818e57-a9c5-41ac-b057-df2fb48e5b86-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gk5b8\" (UID: \"eb818e57-a9c5-41ac-b057-df2fb48e5b86\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gk5b8"
Sep 11 01:08:53.460271 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:53.460133    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb818e57-a9c5-41ac-b057-df2fb48e5b86-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gk5b8\" (UID: \"eb818e57-a9c5-41ac-b057-df2fb48e5b86\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gk5b8"
Sep 11 01:08:53.460271 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:53.460165    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb818e57-a9c5-41ac-b057-df2fb48e5b86-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gk5b8\" (UID: \"eb818e57-a9c5-41ac-b057-df2fb48e5b86\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gk5b8"
Sep 11 01:08:53.460496 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:53.460474    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb818e57-a9c5-41ac-b057-df2fb48e5b86-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gk5b8\" (UID: \"eb818e57-a9c5-41ac-b057-df2fb48e5b86\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gk5b8"
Sep 11 01:08:53.460550 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:53.460501    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb818e57-a9c5-41ac-b057-df2fb48e5b86-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gk5b8\" (UID: \"eb818e57-a9c5-41ac-b057-df2fb48e5b86\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gk5b8"
Sep 11 01:08:53.460585 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:53.460561    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb818e57-a9c5-41ac-b057-df2fb48e5b86-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gk5b8\" (UID: \"eb818e57-a9c5-41ac-b057-df2fb48e5b86\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gk5b8"
Sep 11 01:08:53.462569 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:53.462549    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb818e57-a9c5-41ac-b057-df2fb48e5b86-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gk5b8\" (UID: \"eb818e57-a9c5-41ac-b057-df2fb48e5b86\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gk5b8"
Sep 11 01:08:53.462672 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:53.462650    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb818e57-a9c5-41ac-b057-df2fb48e5b86-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gk5b8\" (UID: \"eb818e57-a9c5-41ac-b057-df2fb48e5b86\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gk5b8"
Sep 11 01:08:53.467882 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:53.467864    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpz8p\" (UniqueName: \"kubernetes.io/projected/eb818e57-a9c5-41ac-b057-df2fb48e5b86-kube-api-access-dpz8p\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gk5b8\" (UID: \"eb818e57-a9c5-41ac-b057-df2fb48e5b86\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gk5b8"
Sep 11 01:08:53.594513 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:53.594421    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gk5b8"
Sep 11 01:08:53.864078 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:53.863808    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-gk5b8"]
Sep 11 01:08:53.866925 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:08:53.866882    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb818e57_a9c5_41ac_b057_df2fb48e5b86.slice/crio-4bd238d6187ffc204a0b1532e4cea50bfd5cee494ac0ce3dc9433ad9e212ead7 WatchSource:0}: Error finding container 4bd238d6187ffc204a0b1532e4cea50bfd5cee494ac0ce3dc9433ad9e212ead7: Status 404 returned error can't find the container with id 4bd238d6187ffc204a0b1532e4cea50bfd5cee494ac0ce3dc9433ad9e212ead7
Sep 11 01:08:53.868547 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:53.868529    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 01:08:53.880932 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:53.880872    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gk5b8" event={"ID":"eb818e57-a9c5-41ac-b057-df2fb48e5b86","Type":"ContainerStarted","Data":"4bd238d6187ffc204a0b1532e4cea50bfd5cee494ac0ce3dc9433ad9e212ead7"}
Sep 11 01:08:54.887248 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:54.887218    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gk5b8" event={"ID":"eb818e57-a9c5-41ac-b057-df2fb48e5b86","Type":"ContainerStarted","Data":"aae4a2975edc1621b50d125c60a29e8f74b72b8b87d717efb5f211f369c72d05"}
Sep 11 01:08:54.903368 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:08:54.903322    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gk5b8" podStartSLOduration=0.945858511 podStartE2EDuration="1.903303708s" totalImagesPullingTime="957.445197ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:08:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:08:54.902416737 +0000 UTC m=+650.299608923" watchObservedRunningTime="2026-09-11 01:08:54.903303708 +0000 UTC m=+650.300495880"
Sep 11 01:09:03.595215 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:03.595175    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gk5b8"
Sep 11 01:09:03.595700 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:03.595246    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gk5b8"
Sep 11 01:09:03.607787 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:03.607764    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gk5b8"
Sep 11 01:09:03.937942 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:03.937889    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gk5b8"
Sep 11 01:09:04.175698 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:04.175658    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v"]
Sep 11 01:09:04.179815 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:04.179796    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v"
Sep 11 01:09:04.182214 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:04.182189    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 11 01:09:04.190385 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:04.190336    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v"]
Sep 11 01:09:04.215807 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:04.215779    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:09:04.355145 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:04.355108    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f72f5638-fbb0-423a-8620-994a1529bf99-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v\" (UID: \"f72f5638-fbb0-423a-8620-994a1529bf99\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v"
Sep 11 01:09:04.355324 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:04.355167    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f72f5638-fbb0-423a-8620-994a1529bf99-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v\" (UID: \"f72f5638-fbb0-423a-8620-994a1529bf99\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v"
Sep 11 01:09:04.355324 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:04.355210    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f72f5638-fbb0-423a-8620-994a1529bf99-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v\" (UID: \"f72f5638-fbb0-423a-8620-994a1529bf99\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v"
Sep 11 01:09:04.355324 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:04.355235    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f72f5638-fbb0-423a-8620-994a1529bf99-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v\" (UID: \"f72f5638-fbb0-423a-8620-994a1529bf99\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v"
Sep 11 01:09:04.355425 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:04.355322    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f72f5638-fbb0-423a-8620-994a1529bf99-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v\" (UID: \"f72f5638-fbb0-423a-8620-994a1529bf99\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v"
Sep 11 01:09:04.355425 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:04.355353    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l9lm\" (UniqueName: \"kubernetes.io/projected/f72f5638-fbb0-423a-8620-994a1529bf99-kube-api-access-7l9lm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v\" (UID: \"f72f5638-fbb0-423a-8620-994a1529bf99\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v"
Sep 11 01:09:04.456206 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:04.456117    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f72f5638-fbb0-423a-8620-994a1529bf99-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v\" (UID: \"f72f5638-fbb0-423a-8620-994a1529bf99\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v"
Sep 11 01:09:04.456206 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:04.456174    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f72f5638-fbb0-423a-8620-994a1529bf99-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v\" (UID: \"f72f5638-fbb0-423a-8620-994a1529bf99\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v"
Sep 11 01:09:04.456206 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:04.456199    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f72f5638-fbb0-423a-8620-994a1529bf99-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v\" (UID: \"f72f5638-fbb0-423a-8620-994a1529bf99\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v"
Sep 11 01:09:04.456421 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:04.456221    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f72f5638-fbb0-423a-8620-994a1529bf99-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v\" (UID: \"f72f5638-fbb0-423a-8620-994a1529bf99\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v"
Sep 11 01:09:04.456421 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:04.456250    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f72f5638-fbb0-423a-8620-994a1529bf99-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v\" (UID: \"f72f5638-fbb0-423a-8620-994a1529bf99\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v"
Sep 11 01:09:04.456421 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:04.456265    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l9lm\" (UniqueName: \"kubernetes.io/projected/f72f5638-fbb0-423a-8620-994a1529bf99-kube-api-access-7l9lm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v\" (UID: \"f72f5638-fbb0-423a-8620-994a1529bf99\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v"
Sep 11 01:09:04.456600 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:04.456576    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f72f5638-fbb0-423a-8620-994a1529bf99-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v\" (UID: \"f72f5638-fbb0-423a-8620-994a1529bf99\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v"
Sep 11 01:09:04.456663 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:04.456633    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f72f5638-fbb0-423a-8620-994a1529bf99-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v\" (UID: \"f72f5638-fbb0-423a-8620-994a1529bf99\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v"
Sep 11 01:09:04.456712 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:04.456666    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f72f5638-fbb0-423a-8620-994a1529bf99-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v\" (UID: \"f72f5638-fbb0-423a-8620-994a1529bf99\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v"
Sep 11 01:09:04.458480 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:04.458451    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f72f5638-fbb0-423a-8620-994a1529bf99-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v\" (UID: \"f72f5638-fbb0-423a-8620-994a1529bf99\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v"
Sep 11 01:09:04.458763 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:04.458746    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f72f5638-fbb0-423a-8620-994a1529bf99-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v\" (UID: \"f72f5638-fbb0-423a-8620-994a1529bf99\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v"
Sep 11 01:09:04.465146 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:04.465124    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l9lm\" (UniqueName: \"kubernetes.io/projected/f72f5638-fbb0-423a-8620-994a1529bf99-kube-api-access-7l9lm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v\" (UID: \"f72f5638-fbb0-423a-8620-994a1529bf99\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v"
Sep 11 01:09:04.491097 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:04.491069    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v"
Sep 11 01:09:04.763270 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:04.763115    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v"]
Sep 11 01:09:04.765934 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:09:04.765892    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf72f5638_fbb0_423a_8620_994a1529bf99.slice/crio-674fb6e06aa3364d2d7bf5fa7f2ab22bdeebc62ae6a4d5d5aa78fdfb24006383 WatchSource:0}: Error finding container 674fb6e06aa3364d2d7bf5fa7f2ab22bdeebc62ae6a4d5d5aa78fdfb24006383: Status 404 returned error can't find the container with id 674fb6e06aa3364d2d7bf5fa7f2ab22bdeebc62ae6a4d5d5aa78fdfb24006383
Sep 11 01:09:04.932002 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:04.931966    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v" event={"ID":"f72f5638-fbb0-423a-8620-994a1529bf99","Type":"ContainerStarted","Data":"674fb6e06aa3364d2d7bf5fa7f2ab22bdeebc62ae6a4d5d5aa78fdfb24006383"}
Sep 11 01:09:05.937667 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:05.937624    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v" event={"ID":"f72f5638-fbb0-423a-8620-994a1529bf99","Type":"ContainerStarted","Data":"6fc895b2e05e63525a4e28bb10eeec0371e0c1d03c77f1266c48079cc44684ce"}
Sep 11 01:09:05.960704 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:05.960660    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v" podStartSLOduration=1.77706532 podStartE2EDuration="1.960641038s" totalImagesPullingTime="183.575718ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:09:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:09:05.959089615 +0000 UTC m=+661.356281787" watchObservedRunningTime="2026-09-11 01:09:05.960641038 +0000 UTC m=+661.357833213"
Sep 11 01:09:14.491362 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:14.491324    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v"
Sep 11 01:09:14.491828 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:14.491412    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v"
Sep 11 01:09:14.503596 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:14.503564    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v"
Sep 11 01:09:14.985581 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:14.985544    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v"
Sep 11 01:09:17.001061 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:17.001023    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:09:40.956321 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:40.956287    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:09:52.522318 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:09:52.522282    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:10:02.618681 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:10:02.618588    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:11:56.347317 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:11:56.347279    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6db4d697d8-ljjlp"]
Sep 11 01:11:56.349430 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:11:56.349410    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6db4d697d8-ljjlp"
Sep 11 01:11:56.352251 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:11:56.352224    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rg9lb\""
Sep 11 01:11:56.352414 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:11:56.352252    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 11 01:11:56.364522 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:11:56.364498    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6db4d697d8-ljjlp"]
Sep 11 01:11:56.398721 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:11:56.398682    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8jws\" (UniqueName: \"kubernetes.io/projected/71578834-02d3-4bc8-ba21-c911a82c1ab0-kube-api-access-j8jws\") pod \"authorino-6db4d697d8-ljjlp\" (UID: \"71578834-02d3-4bc8-ba21-c911a82c1ab0\") " pod="kuadrant-system/authorino-6db4d697d8-ljjlp"
Sep 11 01:11:56.398876 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:11:56.398754    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/71578834-02d3-4bc8-ba21-c911a82c1ab0-tls-cert\") pod \"authorino-6db4d697d8-ljjlp\" (UID: \"71578834-02d3-4bc8-ba21-c911a82c1ab0\") " pod="kuadrant-system/authorino-6db4d697d8-ljjlp"
Sep 11 01:11:56.499314 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:11:56.499274    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/71578834-02d3-4bc8-ba21-c911a82c1ab0-tls-cert\") pod \"authorino-6db4d697d8-ljjlp\" (UID: \"71578834-02d3-4bc8-ba21-c911a82c1ab0\") " pod="kuadrant-system/authorino-6db4d697d8-ljjlp"
Sep 11 01:11:56.499479 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:11:56.499341    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8jws\" (UniqueName: \"kubernetes.io/projected/71578834-02d3-4bc8-ba21-c911a82c1ab0-kube-api-access-j8jws\") pod \"authorino-6db4d697d8-ljjlp\" (UID: \"71578834-02d3-4bc8-ba21-c911a82c1ab0\") " pod="kuadrant-system/authorino-6db4d697d8-ljjlp"
Sep 11 01:11:56.501852 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:11:56.501831    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/71578834-02d3-4bc8-ba21-c911a82c1ab0-tls-cert\") pod \"authorino-6db4d697d8-ljjlp\" (UID: \"71578834-02d3-4bc8-ba21-c911a82c1ab0\") " pod="kuadrant-system/authorino-6db4d697d8-ljjlp"
Sep 11 01:11:56.506811 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:11:56.506793    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8jws\" (UniqueName: \"kubernetes.io/projected/71578834-02d3-4bc8-ba21-c911a82c1ab0-kube-api-access-j8jws\") pod \"authorino-6db4d697d8-ljjlp\" (UID: \"71578834-02d3-4bc8-ba21-c911a82c1ab0\") " pod="kuadrant-system/authorino-6db4d697d8-ljjlp"
Sep 11 01:11:56.657844 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:11:56.657761    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6db4d697d8-ljjlp"
Sep 11 01:11:56.925407 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:11:56.925236    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6db4d697d8-ljjlp"]
Sep 11 01:11:56.928402 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:11:56.928374    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod71578834_02d3_4bc8_ba21_c911a82c1ab0.slice/crio-49c11a0255ab4f82bd70b8ca44a29c53c94bb316d9c30ae562063ed22f615c36 WatchSource:0}: Error finding container 49c11a0255ab4f82bd70b8ca44a29c53c94bb316d9c30ae562063ed22f615c36: Status 404 returned error can't find the container with id 49c11a0255ab4f82bd70b8ca44a29c53c94bb316d9c30ae562063ed22f615c36
Sep 11 01:11:57.676430 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:11:57.676392    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6db4d697d8-ljjlp" event={"ID":"71578834-02d3-4bc8-ba21-c911a82c1ab0","Type":"ContainerStarted","Data":"a804f1e28fc08dc34ee8a60bf806240ea7e5647ca41442d2604c459544c341c0"}
Sep 11 01:11:57.676795 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:11:57.676440    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6db4d697d8-ljjlp" event={"ID":"71578834-02d3-4bc8-ba21-c911a82c1ab0","Type":"ContainerStarted","Data":"49c11a0255ab4f82bd70b8ca44a29c53c94bb316d9c30ae562063ed22f615c36"}
Sep 11 01:11:57.692584 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:11:57.692524    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6db4d697d8-ljjlp" podStartSLOduration=1.146540248 podStartE2EDuration="1.692507275s" totalImagesPullingTime="545.967027ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:11:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:11:57.69011219 +0000 UTC m=+833.087304368" watchObservedRunningTime="2026-09-11 01:11:57.692507275 +0000 UTC m=+833.089699447"
Sep 11 01:11:58.360954 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:11:58.360896    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:11:58.463324 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:11:58.463294    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:11:59.530947 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:11:59.530889    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:11:59.623622 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:11:59.623579    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:16.150797 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:16.150755    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:16.251707 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:16.251672    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:16.359355 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:16.359319    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:16.538132 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:16.538096    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:16.623285 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:16.623242    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:16.713301 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:16.713270    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:16.932638 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:16.932600    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:17.031317 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:17.031282    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:17.131844 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:17.131801    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:17.829445 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:17.829411    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:17.932594 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:17.932563    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:18.351264 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:18.351231    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:18.378020 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:18.377988    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7974-15ff34"]
Sep 11 01:12:18.380223 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:18.380207    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7974-15ff34"
Sep 11 01:12:18.382506 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:18.382479    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-7gdtv\""
Sep 11 01:12:18.386609 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:18.386589    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7974-15ff34"]
Sep 11 01:12:18.487688 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:18.487649    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk6x4\" (UniqueName: \"kubernetes.io/projected/8f339115-eab9-43f6-9fe1-19a7bc06883d-kube-api-access-pk6x4\") pod \"test-curl-7974-15ff34\" (UID: \"8f339115-eab9-43f6-9fe1-19a7bc06883d\") " pod="opendatahub/test-curl-7974-15ff34"
Sep 11 01:12:18.588242 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:18.588200    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk6x4\" (UniqueName: \"kubernetes.io/projected/8f339115-eab9-43f6-9fe1-19a7bc06883d-kube-api-access-pk6x4\") pod \"test-curl-7974-15ff34\" (UID: \"8f339115-eab9-43f6-9fe1-19a7bc06883d\") " pod="opendatahub/test-curl-7974-15ff34"
Sep 11 01:12:18.596397 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:18.596366    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk6x4\" (UniqueName: \"kubernetes.io/projected/8f339115-eab9-43f6-9fe1-19a7bc06883d-kube-api-access-pk6x4\") pod \"test-curl-7974-15ff34\" (UID: \"8f339115-eab9-43f6-9fe1-19a7bc06883d\") " pod="opendatahub/test-curl-7974-15ff34"
Sep 11 01:12:18.689854 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:18.689799    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7974-15ff34"
Sep 11 01:12:18.962256 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:18.962219    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7974-15ff34"]
Sep 11 01:12:18.964989 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:12:18.964965    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8f339115_eab9_43f6_9fe1_19a7bc06883d.slice/crio-ef03f1ef2a328d9505fec7bb612fee5ba8b4bc30e231cc66978b7e9472dcae72 WatchSource:0}: Error finding container ef03f1ef2a328d9505fec7bb612fee5ba8b4bc30e231cc66978b7e9472dcae72: Status 404 returned error can't find the container with id ef03f1ef2a328d9505fec7bb612fee5ba8b4bc30e231cc66978b7e9472dcae72
Sep 11 01:12:19.777585 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:19.777555    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7974-15ff34" event={"ID":"8f339115-eab9-43f6-9fe1-19a7bc06883d","Type":"ContainerStarted","Data":"ef03f1ef2a328d9505fec7bb612fee5ba8b4bc30e231cc66978b7e9472dcae72"}
Sep 11 01:12:19.848273 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:19.848240    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:19.958282 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:19.958239    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:20.064689 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:20.064578    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:20.172649 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:20.172614    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:21.198974 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:21.198944    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:21.300950 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:21.300905    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:21.450863 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:21.450828    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:21.790701 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:21.790618    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7974-15ff34" event={"ID":"8f339115-eab9-43f6-9fe1-19a7bc06883d","Type":"ContainerStarted","Data":"a844fdf21265765d265b33c792778f9fcff407e14771e99854bc0fde4262272f"}
Sep 11 01:12:21.805252 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:21.805175    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7974-15ff34" podStartSLOduration=1.694482411 podStartE2EDuration="3.805155593s" totalImagesPullingTime="2.110673182s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:12:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:12:21.804620475 +0000 UTC m=+857.201812649" watchObservedRunningTime="2026-09-11 01:12:21.805155593 +0000 UTC m=+857.202347767"
Sep 11 01:12:21.834802 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:21.834768    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:21.923307 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:21.923275    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:21.928932 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:21.928552    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e98d1e-98bcf4-brhgp"]
Sep 11 01:12:21.932896 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:21.932870    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e98d1e-98bcf4-brhgp"
Sep 11 01:12:21.934843 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:21.934825    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-e98d1e\""
Sep 11 01:12:21.944556 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:21.944534    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e98d1e-98bcf4-brhgp"]
Sep 11 01:12:21.965148 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:21.965125    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e98d1e-65864c5fb5-ggszd"]
Sep 11 01:12:21.968284 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:21.968266    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e98d1e-65864c5fb5-ggszd"
Sep 11 01:12:21.970846 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:21.970830    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-e98d1e-dockercfg-pcbs6\""
Sep 11 01:12:21.986799 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:21.986777    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e98d1e-65864c5fb5-ggszd"]
Sep 11 01:12:22.019270 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.019238    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlb7q\" (UniqueName: \"kubernetes.io/projected/28c374e4-7a51-4884-aa7f-8080b6d30473-kube-api-access-tlb7q\") pod \"payload-pre-processing-e2e-inf-a-e98d1e-98bcf4-brhgp\" (UID: \"28c374e4-7a51-4884-aa7f-8080b6d30473\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e98d1e-98bcf4-brhgp"
Sep 11 01:12:22.019417 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.019379    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28c374e4-7a51-4884-aa7f-8080b6d30473-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-e98d1e-98bcf4-brhgp\" (UID: \"28c374e4-7a51-4884-aa7f-8080b6d30473\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e98d1e-98bcf4-brhgp"
Sep 11 01:12:22.120187 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.120096    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28c374e4-7a51-4884-aa7f-8080b6d30473-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-e98d1e-98bcf4-brhgp\" (UID: \"28c374e4-7a51-4884-aa7f-8080b6d30473\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e98d1e-98bcf4-brhgp"
Sep 11 01:12:22.120187 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.120153    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/45a3629e-56a1-41b3-892f-8b2d7269e996-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-e98d1e-65864c5fb5-ggszd\" (UID: \"45a3629e-56a1-41b3-892f-8b2d7269e996\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e98d1e-65864c5fb5-ggszd"
Sep 11 01:12:22.120187 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.120194    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfg24\" (UniqueName: \"kubernetes.io/projected/45a3629e-56a1-41b3-892f-8b2d7269e996-kube-api-access-gfg24\") pod \"payload-processing-e2e-inf-a-e98d1e-65864c5fb5-ggszd\" (UID: \"45a3629e-56a1-41b3-892f-8b2d7269e996\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e98d1e-65864c5fb5-ggszd"
Sep 11 01:12:22.120456 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.120280    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlb7q\" (UniqueName: \"kubernetes.io/projected/28c374e4-7a51-4884-aa7f-8080b6d30473-kube-api-access-tlb7q\") pod \"payload-pre-processing-e2e-inf-a-e98d1e-98bcf4-brhgp\" (UID: \"28c374e4-7a51-4884-aa7f-8080b6d30473\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e98d1e-98bcf4-brhgp"
Sep 11 01:12:22.120778 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.120760    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28c374e4-7a51-4884-aa7f-8080b6d30473-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-e98d1e-98bcf4-brhgp\" (UID: \"28c374e4-7a51-4884-aa7f-8080b6d30473\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e98d1e-98bcf4-brhgp"
Sep 11 01:12:22.127876 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.127853    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlb7q\" (UniqueName: \"kubernetes.io/projected/28c374e4-7a51-4884-aa7f-8080b6d30473-kube-api-access-tlb7q\") pod \"payload-pre-processing-e2e-inf-a-e98d1e-98bcf4-brhgp\" (UID: \"28c374e4-7a51-4884-aa7f-8080b6d30473\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e98d1e-98bcf4-brhgp"
Sep 11 01:12:22.221625 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.221578    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/45a3629e-56a1-41b3-892f-8b2d7269e996-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-e98d1e-65864c5fb5-ggszd\" (UID: \"45a3629e-56a1-41b3-892f-8b2d7269e996\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e98d1e-65864c5fb5-ggszd"
Sep 11 01:12:22.222014 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.221639    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfg24\" (UniqueName: \"kubernetes.io/projected/45a3629e-56a1-41b3-892f-8b2d7269e996-kube-api-access-gfg24\") pod \"payload-processing-e2e-inf-a-e98d1e-65864c5fb5-ggszd\" (UID: \"45a3629e-56a1-41b3-892f-8b2d7269e996\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e98d1e-65864c5fb5-ggszd"
Sep 11 01:12:22.222384 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.222365    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/45a3629e-56a1-41b3-892f-8b2d7269e996-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-e98d1e-65864c5fb5-ggszd\" (UID: \"45a3629e-56a1-41b3-892f-8b2d7269e996\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e98d1e-65864c5fb5-ggszd"
Sep 11 01:12:22.235045 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.235020    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfg24\" (UniqueName: \"kubernetes.io/projected/45a3629e-56a1-41b3-892f-8b2d7269e996-kube-api-access-gfg24\") pod \"payload-processing-e2e-inf-a-e98d1e-65864c5fb5-ggszd\" (UID: \"45a3629e-56a1-41b3-892f-8b2d7269e996\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e98d1e-65864c5fb5-ggszd"
Sep 11 01:12:22.242771 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.242748    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e98d1e-98bcf4-brhgp"
Sep 11 01:12:22.263253 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.263219    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:22.278070 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.278038    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e98d1e-65864c5fb5-ggszd"
Sep 11 01:12:22.398780 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.398702    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:22.476419 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.476386    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7974-15ff34"]
Sep 11 01:12:22.481226 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.481187    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7974-15ff34"]
Sep 11 01:12:22.527535 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.527506    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e98d1e-98bcf4-brhgp"]
Sep 11 01:12:22.530505 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:12:22.530471    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28c374e4_7a51_4884_aa7f_8080b6d30473.slice/crio-7c6701e0c8712fda0d36da089a747f9af5ac9ed8626f99095b6c3ae4c90705c7 WatchSource:0}: Error finding container 7c6701e0c8712fda0d36da089a747f9af5ac9ed8626f99095b6c3ae4c90705c7: Status 404 returned error can't find the container with id 7c6701e0c8712fda0d36da089a747f9af5ac9ed8626f99095b6c3ae4c90705c7
Sep 11 01:12:22.565037 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.565014    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e98d1e-65864c5fb5-ggszd"]
Sep 11 01:12:22.567470 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:12:22.567443    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45a3629e_56a1_41b3_892f_8b2d7269e996.slice/crio-f4e3632b40092821fd1224c86619c29a348ba047216f25904c12d8b285099b78 WatchSource:0}: Error finding container f4e3632b40092821fd1224c86619c29a348ba047216f25904c12d8b285099b78: Status 404 returned error can't find the container with id f4e3632b40092821fd1224c86619c29a348ba047216f25904c12d8b285099b78
Sep 11 01:12:22.617735 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.617711    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7974-00f22f"]
Sep 11 01:12:22.618120 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.618104    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f339115-eab9-43f6-9fe1-19a7bc06883d" containerName="test-curl-7974-15ff34"
Sep 11 01:12:22.618120 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.618122    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f339115-eab9-43f6-9fe1-19a7bc06883d" containerName="test-curl-7974-15ff34"
Sep 11 01:12:22.618302 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.618175    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f339115-eab9-43f6-9fe1-19a7bc06883d" containerName="test-curl-7974-15ff34"
Sep 11 01:12:22.621201 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.621181    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7974-00f22f"
Sep 11 01:12:22.626984 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.626957    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7974-00f22f"]
Sep 11 01:12:22.726078 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.726035    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgrrn\" (UniqueName: \"kubernetes.io/projected/2a2c43c3-ebf7-4f61-a1d2-e765fc742ee6-kube-api-access-wgrrn\") pod \"test-curl-7974-00f22f\" (UID: \"2a2c43c3-ebf7-4f61-a1d2-e765fc742ee6\") " pod="opendatahub/test-curl-7974-00f22f"
Sep 11 01:12:22.798650 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.798614    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e98d1e-98bcf4-brhgp" event={"ID":"28c374e4-7a51-4884-aa7f-8080b6d30473","Type":"ContainerStarted","Data":"78c81e3c5ff97d6813c3516509b7aac909071720a22d48068957f5a06d32f2bc"}
Sep 11 01:12:22.798882 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.798856    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e98d1e-98bcf4-brhgp"
Sep 11 01:12:22.799044 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.798889    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e98d1e-98bcf4-brhgp" event={"ID":"28c374e4-7a51-4884-aa7f-8080b6d30473","Type":"ContainerStarted","Data":"7c6701e0c8712fda0d36da089a747f9af5ac9ed8626f99095b6c3ae4c90705c7"}
Sep 11 01:12:22.800395 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.800359    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e98d1e-65864c5fb5-ggszd" event={"ID":"45a3629e-56a1-41b3-892f-8b2d7269e996","Type":"ContainerStarted","Data":"fb01b84ebb7eaeb9c459aca71eb842c7c6707ed4e13f86c55e4aed3cd0d6ec0c"}
Sep 11 01:12:22.800495 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.800407    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e98d1e-65864c5fb5-ggszd" event={"ID":"45a3629e-56a1-41b3-892f-8b2d7269e996","Type":"ContainerStarted","Data":"f4e3632b40092821fd1224c86619c29a348ba047216f25904c12d8b285099b78"}
Sep 11 01:12:22.800558 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:12:22.800515    2574 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7974-15ff34" container="test-curl-7974-15ff34"
Sep 11 01:12:22.815070 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.815023    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e98d1e-98bcf4-brhgp" podStartSLOduration=1.815004463 podStartE2EDuration="1.815004463s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:12:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:12:22.814184775 +0000 UTC m=+858.211376951" watchObservedRunningTime="2026-09-11 01:12:22.815004463 +0000 UTC m=+858.212196636"
Sep 11 01:12:22.827024 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.826997    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgrrn\" (UniqueName: \"kubernetes.io/projected/2a2c43c3-ebf7-4f61-a1d2-e765fc742ee6-kube-api-access-wgrrn\") pod \"test-curl-7974-00f22f\" (UID: \"2a2c43c3-ebf7-4f61-a1d2-e765fc742ee6\") " pod="opendatahub/test-curl-7974-00f22f"
Sep 11 01:12:22.829226 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.829188    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-e98d1e-65864c5fb5-ggszd" podStartSLOduration=1.829171858 podStartE2EDuration="1.829171858s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:12:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:12:22.829016902 +0000 UTC m=+858.226209076" watchObservedRunningTime="2026-09-11 01:12:22.829171858 +0000 UTC m=+858.226364012"
Sep 11 01:12:22.838550 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.838529    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgrrn\" (UniqueName: \"kubernetes.io/projected/2a2c43c3-ebf7-4f61-a1d2-e765fc742ee6-kube-api-access-wgrrn\") pod \"test-curl-7974-00f22f\" (UID: \"2a2c43c3-ebf7-4f61-a1d2-e765fc742ee6\") " pod="opendatahub/test-curl-7974-00f22f"
Sep 11 01:12:22.932754 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.932727    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7974-00f22f"
Sep 11 01:12:22.950249 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:22.950219    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:23.048619 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.048591    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:23.068513 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.068479    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-f4de10-977ff8d6-6g49f"]
Sep 11 01:12:23.072660 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.072641    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f4de10-977ff8d6-6g49f"
Sep 11 01:12:23.075064 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.075036    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-f4de10-dockercfg-d56x7\""
Sep 11 01:12:23.075174 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.075130    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-f4de10\""
Sep 11 01:12:23.084563 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.084539    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-f4de10-977ff8d6-6g49f"]
Sep 11 01:12:23.101312 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.101289    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-f4de10-857b967875-n67cq"]
Sep 11 01:12:23.104565 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.104547    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-f4de10-857b967875-n67cq"
Sep 11 01:12:23.118509 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.118477    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-f4de10-857b967875-n67cq"]
Sep 11 01:12:23.211643 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.211292    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7974-00f22f"]
Sep 11 01:12:23.214774 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:12:23.214748    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2a2c43c3_ebf7_4f61_a1d2_e765fc742ee6.slice/crio-3513cf72a9cafd10fb612a5bf90b1121824d31a6656769dfa74263e71ae33b4d WatchSource:0}: Error finding container 3513cf72a9cafd10fb612a5bf90b1121824d31a6656769dfa74263e71ae33b4d: Status 404 returned error can't find the container with id 3513cf72a9cafd10fb612a5bf90b1121824d31a6656769dfa74263e71ae33b4d
Sep 11 01:12:23.231507 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.231474    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4337d7d-b9d3-43f4-9cb8-5ae74419c273-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-f4de10-857b967875-n67cq\" (UID: \"c4337d7d-b9d3-43f4-9cb8-5ae74419c273\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-f4de10-857b967875-n67cq"
Sep 11 01:12:23.231892 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.231525    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c67d64bb-40f3-4540-a3a0-8001a339a21c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-f4de10-977ff8d6-6g49f\" (UID: \"c67d64bb-40f3-4540-a3a0-8001a339a21c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f4de10-977ff8d6-6g49f"
Sep 11 01:12:23.231892 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.231632    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ml6c7\" (UniqueName: \"kubernetes.io/projected/c67d64bb-40f3-4540-a3a0-8001a339a21c-kube-api-access-ml6c7\") pod \"payload-pre-processing-e2e-worker-w0-f4de10-977ff8d6-6g49f\" (UID: \"c67d64bb-40f3-4540-a3a0-8001a339a21c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f4de10-977ff8d6-6g49f"
Sep 11 01:12:23.231892 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.231648    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv4db\" (UniqueName: \"kubernetes.io/projected/c4337d7d-b9d3-43f4-9cb8-5ae74419c273-kube-api-access-jv4db\") pod \"payload-processing-e2e-worker-w0-f4de10-857b967875-n67cq\" (UID: \"c4337d7d-b9d3-43f4-9cb8-5ae74419c273\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-f4de10-857b967875-n67cq"
Sep 11 01:12:23.332127 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.332093    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4337d7d-b9d3-43f4-9cb8-5ae74419c273-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-f4de10-857b967875-n67cq\" (UID: \"c4337d7d-b9d3-43f4-9cb8-5ae74419c273\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-f4de10-857b967875-n67cq"
Sep 11 01:12:23.332276 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.332136    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c67d64bb-40f3-4540-a3a0-8001a339a21c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-f4de10-977ff8d6-6g49f\" (UID: \"c67d64bb-40f3-4540-a3a0-8001a339a21c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f4de10-977ff8d6-6g49f"
Sep 11 01:12:23.332276 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.332213    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ml6c7\" (UniqueName: \"kubernetes.io/projected/c67d64bb-40f3-4540-a3a0-8001a339a21c-kube-api-access-ml6c7\") pod \"payload-pre-processing-e2e-worker-w0-f4de10-977ff8d6-6g49f\" (UID: \"c67d64bb-40f3-4540-a3a0-8001a339a21c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f4de10-977ff8d6-6g49f"
Sep 11 01:12:23.332276 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.332236    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv4db\" (UniqueName: \"kubernetes.io/projected/c4337d7d-b9d3-43f4-9cb8-5ae74419c273-kube-api-access-jv4db\") pod \"payload-processing-e2e-worker-w0-f4de10-857b967875-n67cq\" (UID: \"c4337d7d-b9d3-43f4-9cb8-5ae74419c273\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-f4de10-857b967875-n67cq"
Sep 11 01:12:23.332831 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.332811    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4337d7d-b9d3-43f4-9cb8-5ae74419c273-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-f4de10-857b967875-n67cq\" (UID: \"c4337d7d-b9d3-43f4-9cb8-5ae74419c273\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-f4de10-857b967875-n67cq"
Sep 11 01:12:23.333071 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.333048    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c67d64bb-40f3-4540-a3a0-8001a339a21c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-f4de10-977ff8d6-6g49f\" (UID: \"c67d64bb-40f3-4540-a3a0-8001a339a21c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f4de10-977ff8d6-6g49f"
Sep 11 01:12:23.342123 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.342097    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ml6c7\" (UniqueName: \"kubernetes.io/projected/c67d64bb-40f3-4540-a3a0-8001a339a21c-kube-api-access-ml6c7\") pod \"payload-pre-processing-e2e-worker-w0-f4de10-977ff8d6-6g49f\" (UID: \"c67d64bb-40f3-4540-a3a0-8001a339a21c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f4de10-977ff8d6-6g49f"
Sep 11 01:12:23.342300 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.342284    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv4db\" (UniqueName: \"kubernetes.io/projected/c4337d7d-b9d3-43f4-9cb8-5ae74419c273-kube-api-access-jv4db\") pod \"payload-processing-e2e-worker-w0-f4de10-857b967875-n67cq\" (UID: \"c4337d7d-b9d3-43f4-9cb8-5ae74419c273\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-f4de10-857b967875-n67cq"
Sep 11 01:12:23.385144 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.385111    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f4de10-977ff8d6-6g49f"
Sep 11 01:12:23.414505 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.414460    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-f4de10-857b967875-n67cq"
Sep 11 01:12:23.526556 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.526464    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:23.627184 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.627152    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:23.658633 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.658599    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-f4de10-977ff8d6-6g49f"]
Sep 11 01:12:23.661865 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:12:23.661839    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc67d64bb_40f3_4540_a3a0_8001a339a21c.slice/crio-990ce1d1d3a3fb6fd100a13e672529cb34828673667b08255de28530381e1373 WatchSource:0}: Error finding container 990ce1d1d3a3fb6fd100a13e672529cb34828673667b08255de28530381e1373: Status 404 returned error can't find the container with id 990ce1d1d3a3fb6fd100a13e672529cb34828673667b08255de28530381e1373
Sep 11 01:12:23.701668 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.701643    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-f4de10-857b967875-n67cq"]
Sep 11 01:12:23.704070 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:12:23.704045    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4337d7d_b9d3_43f4_9cb8_5ae74419c273.slice/crio-718e9a230a1701a4105c9af769d5646ab256b14c10655aa07217c3114b718c8d WatchSource:0}: Error finding container 718e9a230a1701a4105c9af769d5646ab256b14c10655aa07217c3114b718c8d: Status 404 returned error can't find the container with id 718e9a230a1701a4105c9af769d5646ab256b14c10655aa07217c3114b718c8d
Sep 11 01:12:23.809453 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.808541    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f4de10-977ff8d6-6g49f" event={"ID":"c67d64bb-40f3-4540-a3a0-8001a339a21c","Type":"ContainerStarted","Data":"7dd2ba2d1c951873b3830284a4c85b2642b851f3b5e58a28e7a9ebcfbd521802"}
Sep 11 01:12:23.809453 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.808602    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f4de10-977ff8d6-6g49f" event={"ID":"c67d64bb-40f3-4540-a3a0-8001a339a21c","Type":"ContainerStarted","Data":"990ce1d1d3a3fb6fd100a13e672529cb34828673667b08255de28530381e1373"}
Sep 11 01:12:23.809453 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.809330    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f4de10-977ff8d6-6g49f"
Sep 11 01:12:23.812822 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.811613    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7974-00f22f" event={"ID":"2a2c43c3-ebf7-4f61-a1d2-e765fc742ee6","Type":"ContainerStarted","Data":"d35c0a69fc4e6ce6940148c44efdfdcb10f75e99766b4a2dcd431f719cc61aa1"}
Sep 11 01:12:23.812822 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.811648    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7974-00f22f" event={"ID":"2a2c43c3-ebf7-4f61-a1d2-e765fc742ee6","Type":"ContainerStarted","Data":"3513cf72a9cafd10fb612a5bf90b1121824d31a6656769dfa74263e71ae33b4d"}
Sep 11 01:12:23.816414 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.815584    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-f4de10-857b967875-n67cq" event={"ID":"c4337d7d-b9d3-43f4-9cb8-5ae74419c273","Type":"ContainerStarted","Data":"8021eb8446c812146a5e4314da5efe5a3db20f5e556ba12e3d715e23fec294d5"}
Sep 11 01:12:23.816414 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.815627    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-f4de10-857b967875-n67cq"
Sep 11 01:12:23.816414 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.815648    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-f4de10-857b967875-n67cq" event={"ID":"c4337d7d-b9d3-43f4-9cb8-5ae74419c273","Type":"ContainerStarted","Data":"718e9a230a1701a4105c9af769d5646ab256b14c10655aa07217c3114b718c8d"}
Sep 11 01:12:23.816414 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.816046    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-e98d1e-65864c5fb5-ggszd"
Sep 11 01:12:23.816414 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.816145    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7974-15ff34" podUID="8f339115-eab9-43f6-9fe1-19a7bc06883d" containerName="test-curl-7974-15ff34" containerID="cri-o://a844fdf21265765d265b33c792778f9fcff407e14771e99854bc0fde4262272f" gracePeriod=2
Sep 11 01:12:23.829451 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.829399    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f4de10-977ff8d6-6g49f" podStartSLOduration=0.829382369 podStartE2EDuration="829.382369ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:12:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:12:23.828065258 +0000 UTC m=+859.225257440" watchObservedRunningTime="2026-09-11 01:12:23.829382369 +0000 UTC m=+859.226574544"
Sep 11 01:12:23.846059 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.846018    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7974-00f22f" podStartSLOduration=1.846005253 podStartE2EDuration="1.846005253s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:12:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:12:23.844352357 +0000 UTC m=+859.241544532" watchObservedRunningTime="2026-09-11 01:12:23.846005253 +0000 UTC m=+859.243197425"
Sep 11 01:12:23.861074 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:23.861002    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-f4de10-857b967875-n67cq" podStartSLOduration=0.860988507 podStartE2EDuration="860.988507ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:12:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:12:23.858731537 +0000 UTC m=+859.255923702" watchObservedRunningTime="2026-09-11 01:12:23.860988507 +0000 UTC m=+859.258180681"
Sep 11 01:12:24.089876 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:24.089795    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:24.189299 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:24.189267    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:24.257687 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:24.257659    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:24.262099 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:24.262075    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-cd176e-79cc7879f4-tg9wg"]
Sep 11 01:12:24.265591 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:24.265573    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd176e-79cc7879f4-tg9wg"
Sep 11 01:12:24.267614 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:24.267585    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-cd176e\""
Sep 11 01:12:24.267734 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:24.267690    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-cd176e-dockercfg-pkrtb\""
Sep 11 01:12:24.277131 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:24.277106    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-cd176e-79cc7879f4-tg9wg"]
Sep 11 01:12:24.340355 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:24.340276    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-cd176e-79cc7879f4-tg9wg\" (UID: \"7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd176e-79cc7879f4-tg9wg"
Sep 11 01:12:24.340355 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:24.340330    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxzk5\" (UniqueName: \"kubernetes.io/projected/7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7-kube-api-access-xxzk5\") pod \"payload-pre-processing-e2e-worker-w4-cd176e-79cc7879f4-tg9wg\" (UID: \"7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd176e-79cc7879f4-tg9wg"
Sep 11 01:12:24.440961 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:24.440893    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-cd176e-79cc7879f4-tg9wg\" (UID: \"7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd176e-79cc7879f4-tg9wg"
Sep 11 01:12:24.441121 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:24.440976    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxzk5\" (UniqueName: \"kubernetes.io/projected/7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7-kube-api-access-xxzk5\") pod \"payload-pre-processing-e2e-worker-w4-cd176e-79cc7879f4-tg9wg\" (UID: \"7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd176e-79cc7879f4-tg9wg"
Sep 11 01:12:24.441657 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:24.441637    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-cd176e-79cc7879f4-tg9wg\" (UID: \"7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd176e-79cc7879f4-tg9wg"
Sep 11 01:12:24.448515 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:24.448491    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxzk5\" (UniqueName: \"kubernetes.io/projected/7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7-kube-api-access-xxzk5\") pod \"payload-pre-processing-e2e-worker-w4-cd176e-79cc7879f4-tg9wg\" (UID: \"7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd176e-79cc7879f4-tg9wg"
Sep 11 01:12:24.502531 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:24.502494    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7974-00f22f"]
Sep 11 01:12:24.507609 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:24.507578    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7974-00f22f"]
Sep 11 01:12:24.516623 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:24.516593    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:24.578766 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:24.578736    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd176e-79cc7879f4-tg9wg"
Sep 11 01:12:24.819136 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:12:24.819096    2574 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7974-00f22f" container="test-curl-7974-00f22f"
Sep 11 01:12:24.854085 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:24.854015    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-cd176e-79cc7879f4-tg9wg"]
Sep 11 01:12:24.857001 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:12:24.856975    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eebe6a6_8aca_4f77_b7b8_2ab5f6ee1bf7.slice/crio-591d6ec81fe3d0fbb4de65e56ab80a20c93c6fd732a897ed51f6a80109df39dd WatchSource:0}: Error finding container 591d6ec81fe3d0fbb4de65e56ab80a20c93c6fd732a897ed51f6a80109df39dd: Status 404 returned error can't find the container with id 591d6ec81fe3d0fbb4de65e56ab80a20c93c6fd732a897ed51f6a80109df39dd
Sep 11 01:12:25.082730 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:25.082691    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:25.259704 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:25.259661    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:25.361948 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:25.361893    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:25.457743 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:25.457699    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:25.557106 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:25.557025    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:25.813423 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:25.813341    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:25.826413 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:25.826378    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd176e-79cc7879f4-tg9wg" event={"ID":"7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7","Type":"ContainerStarted","Data":"82ec67455be51090f4b6d0f48fc21064a26443133d0a643cb47e84f010be2d37"}
Sep 11 01:12:25.826535 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:25.826505    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd176e-79cc7879f4-tg9wg"
Sep 11 01:12:25.826591 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:25.826535    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd176e-79cc7879f4-tg9wg" event={"ID":"7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7","Type":"ContainerStarted","Data":"591d6ec81fe3d0fbb4de65e56ab80a20c93c6fd732a897ed51f6a80109df39dd"}
Sep 11 01:12:25.826811 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:25.826764    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7974-00f22f" podUID="2a2c43c3-ebf7-4f61-a1d2-e765fc742ee6" containerName="test-curl-7974-00f22f" containerID="cri-o://d35c0a69fc4e6ce6940148c44efdfdcb10f75e99766b4a2dcd431f719cc61aa1" gracePeriod=2
Sep 11 01:12:25.849468 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:25.849418    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd176e-79cc7879f4-tg9wg" podStartSLOduration=1.849399201 podStartE2EDuration="1.849399201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:12:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:12:25.848018843 +0000 UTC m=+861.245211015" watchObservedRunningTime="2026-09-11 01:12:25.849399201 +0000 UTC m=+861.246591374"
Sep 11 01:12:25.938714 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:25.938682    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:26.002562 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:26.002535    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7974-15ff34"
Sep 11 01:12:26.155752 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:26.155656    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pk6x4\" (UniqueName: \"kubernetes.io/projected/8f339115-eab9-43f6-9fe1-19a7bc06883d-kube-api-access-pk6x4\") pod \"8f339115-eab9-43f6-9fe1-19a7bc06883d\" (UID: \"8f339115-eab9-43f6-9fe1-19a7bc06883d\") "
Sep 11 01:12:26.158009 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:26.157987    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f339115-eab9-43f6-9fe1-19a7bc06883d-kube-api-access-pk6x4" (OuterVolumeSpecName: "kube-api-access-pk6x4") pod "8f339115-eab9-43f6-9fe1-19a7bc06883d" (UID: "8f339115-eab9-43f6-9fe1-19a7bc06883d"). InnerVolumeSpecName "kube-api-access-pk6x4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:12:26.231996 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:26.231957    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:26.257177 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:26.257144    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pk6x4\" (UniqueName: \"kubernetes.io/projected/8f339115-eab9-43f6-9fe1-19a7bc06883d-kube-api-access-pk6x4\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:12:26.832628 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:26.832592    2574 generic.go:356] "Generic (PLEG): container finished" podID="8f339115-eab9-43f6-9fe1-19a7bc06883d" containerID="a844fdf21265765d265b33c792778f9fcff407e14771e99854bc0fde4262272f" exitCode=137
Sep 11 01:12:26.832628 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:26.832633    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7974-15ff34"
Sep 11 01:12:26.833156 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:26.832691    2574 scope.go:117] "RemoveContainer" containerID="a844fdf21265765d265b33c792778f9fcff407e14771e99854bc0fde4262272f"
Sep 11 01:12:26.841348 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:26.841329    2574 scope.go:117] "RemoveContainer" containerID="a844fdf21265765d265b33c792778f9fcff407e14771e99854bc0fde4262272f"
Sep 11 01:12:26.841643 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:12:26.841624    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a844fdf21265765d265b33c792778f9fcff407e14771e99854bc0fde4262272f\": container with ID starting with a844fdf21265765d265b33c792778f9fcff407e14771e99854bc0fde4262272f not found: ID does not exist" containerID="a844fdf21265765d265b33c792778f9fcff407e14771e99854bc0fde4262272f"
Sep 11 01:12:26.841702 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:26.841656    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a844fdf21265765d265b33c792778f9fcff407e14771e99854bc0fde4262272f"} err="failed to get container status \"a844fdf21265765d265b33c792778f9fcff407e14771e99854bc0fde4262272f\": rpc error: code = NotFound desc = could not find container \"a844fdf21265765d265b33c792778f9fcff407e14771e99854bc0fde4262272f\": container with ID starting with a844fdf21265765d265b33c792778f9fcff407e14771e99854bc0fde4262272f not found: ID does not exist"
Sep 11 01:12:26.843671 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:12:26.843649    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7974-15ff34\" is forbidden: User \"system:node:ip-10-0-139-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-235.ec2.internal' and this object" podUID="8f339115-eab9-43f6-9fe1-19a7bc06883d" pod="opendatahub/test-curl-7974-15ff34"
Sep 11 01:12:27.108315 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:27.108231    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f339115-eab9-43f6-9fe1-19a7bc06883d" path="/var/lib/kubelet/pods/8f339115-eab9-43f6-9fe1-19a7bc06883d/volumes"
Sep 11 01:12:28.017743 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:28.017714    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7974-00f22f"
Sep 11 01:12:28.174521 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:28.174483    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wgrrn\" (UniqueName: \"kubernetes.io/projected/2a2c43c3-ebf7-4f61-a1d2-e765fc742ee6-kube-api-access-wgrrn\") pod \"2a2c43c3-ebf7-4f61-a1d2-e765fc742ee6\" (UID: \"2a2c43c3-ebf7-4f61-a1d2-e765fc742ee6\") "
Sep 11 01:12:28.176731 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:28.176701    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a2c43c3-ebf7-4f61-a1d2-e765fc742ee6-kube-api-access-wgrrn" (OuterVolumeSpecName: "kube-api-access-wgrrn") pod "2a2c43c3-ebf7-4f61-a1d2-e765fc742ee6" (UID: "2a2c43c3-ebf7-4f61-a1d2-e765fc742ee6"). InnerVolumeSpecName "kube-api-access-wgrrn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:12:28.275620 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:28.275584    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wgrrn\" (UniqueName: \"kubernetes.io/projected/2a2c43c3-ebf7-4f61-a1d2-e765fc742ee6-kube-api-access-wgrrn\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:12:28.792811 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:28.792775    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:28.845996 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:28.845951    2574 generic.go:356] "Generic (PLEG): container finished" podID="2a2c43c3-ebf7-4f61-a1d2-e765fc742ee6" containerID="d35c0a69fc4e6ce6940148c44efdfdcb10f75e99766b4a2dcd431f719cc61aa1" exitCode=137
Sep 11 01:12:28.845996 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:28.845990    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7974-00f22f"
Sep 11 01:12:28.846264 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:28.846021    2574 scope.go:117] "RemoveContainer" containerID="d35c0a69fc4e6ce6940148c44efdfdcb10f75e99766b4a2dcd431f719cc61aa1"
Sep 11 01:12:28.855879 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:28.855857    2574 scope.go:117] "RemoveContainer" containerID="d35c0a69fc4e6ce6940148c44efdfdcb10f75e99766b4a2dcd431f719cc61aa1"
Sep 11 01:12:28.856249 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:12:28.856225    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d35c0a69fc4e6ce6940148c44efdfdcb10f75e99766b4a2dcd431f719cc61aa1\": container with ID starting with d35c0a69fc4e6ce6940148c44efdfdcb10f75e99766b4a2dcd431f719cc61aa1 not found: ID does not exist" containerID="d35c0a69fc4e6ce6940148c44efdfdcb10f75e99766b4a2dcd431f719cc61aa1"
Sep 11 01:12:28.856398 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:28.856263    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d35c0a69fc4e6ce6940148c44efdfdcb10f75e99766b4a2dcd431f719cc61aa1"} err="failed to get container status \"d35c0a69fc4e6ce6940148c44efdfdcb10f75e99766b4a2dcd431f719cc61aa1\": rpc error: code = NotFound desc = could not find container \"d35c0a69fc4e6ce6940148c44efdfdcb10f75e99766b4a2dcd431f719cc61aa1\": container with ID starting with d35c0a69fc4e6ce6940148c44efdfdcb10f75e99766b4a2dcd431f719cc61aa1 not found: ID does not exist"
Sep 11 01:12:28.857008 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:12:28.856983    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7974-00f22f\" is forbidden: User \"system:node:ip-10-0-139-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-235.ec2.internal' and this object" podUID="2a2c43c3-ebf7-4f61-a1d2-e765fc742ee6" pod="opendatahub/test-curl-7974-00f22f"
Sep 11 01:12:28.891676 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:28.891644    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:29.100063 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:29.099983    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a2c43c3-ebf7-4f61-a1d2-e765fc742ee6" path="/var/lib/kubelet/pods/2a2c43c3-ebf7-4f61-a1d2-e765fc742ee6/volumes"
Sep 11 01:12:30.474239 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:30.474205    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:30.578898 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:30.578852    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:30.925815 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:30.925788    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:31.229347 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:31.229259    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:31.344111 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:31.344070    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:31.444266 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:31.444233    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2c1e69-6cd8cbffb6-k5pwp"]
Sep 11 01:12:31.444603 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:31.444591    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a2c43c3-ebf7-4f61-a1d2-e765fc742ee6" containerName="test-curl-7974-00f22f"
Sep 11 01:12:31.444653 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:31.444605    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a2c43c3-ebf7-4f61-a1d2-e765fc742ee6" containerName="test-curl-7974-00f22f"
Sep 11 01:12:31.444686 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:31.444673    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a2c43c3-ebf7-4f61-a1d2-e765fc742ee6" containerName="test-curl-7974-00f22f"
Sep 11 01:12:31.447235 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:31.447219    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-2c1e69-6cd8cbffb6-k5pwp"
Sep 11 01:12:31.449903 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:31.449878    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-2c1e69\""
Sep 11 01:12:31.450051 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:31.449987    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-2c1e69-dockercfg-n4zlp\""
Sep 11 01:12:31.460511 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:31.460488    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2c1e69-6cd8cbffb6-k5pwp"]
Sep 11 01:12:31.520730 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:31.520657    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-2c1e69-6ff7b679df-nvthr"]
Sep 11 01:12:31.524157 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:31.524142    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-2c1e69-6ff7b679df-nvthr"
Sep 11 01:12:31.535243 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:31.535222    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-2c1e69-6ff7b679df-nvthr"]
Sep 11 01:12:31.607355 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:31.607321    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93b6d88b-42f3-44ae-b042-54e5f8e0f763-plugins-config-volume\") pod \"payload-processing-e2e-autores-2c1e69-6ff7b679df-nvthr\" (UID: \"93b6d88b-42f3-44ae-b042-54e5f8e0f763\") " pod="openshift-ingress/payload-processing-e2e-autores-2c1e69-6ff7b679df-nvthr"
Sep 11 01:12:31.607355 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:31.607369    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb5br\" (UniqueName: \"kubernetes.io/projected/af08e083-3649-4feb-8f36-2eb01da9a3db-kube-api-access-kb5br\") pod \"payload-pre-processing-e2e-autores-2c1e69-6cd8cbffb6-k5pwp\" (UID: \"af08e083-3649-4feb-8f36-2eb01da9a3db\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2c1e69-6cd8cbffb6-k5pwp"
Sep 11 01:12:31.607589 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:31.607424    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swbb6\" (UniqueName: \"kubernetes.io/projected/93b6d88b-42f3-44ae-b042-54e5f8e0f763-kube-api-access-swbb6\") pod \"payload-processing-e2e-autores-2c1e69-6ff7b679df-nvthr\" (UID: \"93b6d88b-42f3-44ae-b042-54e5f8e0f763\") " pod="openshift-ingress/payload-processing-e2e-autores-2c1e69-6ff7b679df-nvthr"
Sep 11 01:12:31.607589 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:31.607468    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af08e083-3649-4feb-8f36-2eb01da9a3db-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-2c1e69-6cd8cbffb6-k5pwp\" (UID: \"af08e083-3649-4feb-8f36-2eb01da9a3db\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2c1e69-6cd8cbffb6-k5pwp"
Sep 11 01:12:31.708788 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:31.708754    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93b6d88b-42f3-44ae-b042-54e5f8e0f763-plugins-config-volume\") pod \"payload-processing-e2e-autores-2c1e69-6ff7b679df-nvthr\" (UID: \"93b6d88b-42f3-44ae-b042-54e5f8e0f763\") " pod="openshift-ingress/payload-processing-e2e-autores-2c1e69-6ff7b679df-nvthr"
Sep 11 01:12:31.708990 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:31.708804    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb5br\" (UniqueName: \"kubernetes.io/projected/af08e083-3649-4feb-8f36-2eb01da9a3db-kube-api-access-kb5br\") pod \"payload-pre-processing-e2e-autores-2c1e69-6cd8cbffb6-k5pwp\" (UID: \"af08e083-3649-4feb-8f36-2eb01da9a3db\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2c1e69-6cd8cbffb6-k5pwp"
Sep 11 01:12:31.708990 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:31.708842    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swbb6\" (UniqueName: \"kubernetes.io/projected/93b6d88b-42f3-44ae-b042-54e5f8e0f763-kube-api-access-swbb6\") pod \"payload-processing-e2e-autores-2c1e69-6ff7b679df-nvthr\" (UID: \"93b6d88b-42f3-44ae-b042-54e5f8e0f763\") " pod="openshift-ingress/payload-processing-e2e-autores-2c1e69-6ff7b679df-nvthr"
Sep 11 01:12:31.708990 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:31.708892    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af08e083-3649-4feb-8f36-2eb01da9a3db-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-2c1e69-6cd8cbffb6-k5pwp\" (UID: \"af08e083-3649-4feb-8f36-2eb01da9a3db\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2c1e69-6cd8cbffb6-k5pwp"
Sep 11 01:12:31.709523 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:31.709501    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93b6d88b-42f3-44ae-b042-54e5f8e0f763-plugins-config-volume\") pod \"payload-processing-e2e-autores-2c1e69-6ff7b679df-nvthr\" (UID: \"93b6d88b-42f3-44ae-b042-54e5f8e0f763\") " pod="openshift-ingress/payload-processing-e2e-autores-2c1e69-6ff7b679df-nvthr"
Sep 11 01:12:31.709579 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:31.709565    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af08e083-3649-4feb-8f36-2eb01da9a3db-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-2c1e69-6cd8cbffb6-k5pwp\" (UID: \"af08e083-3649-4feb-8f36-2eb01da9a3db\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2c1e69-6cd8cbffb6-k5pwp"
Sep 11 01:12:31.716569 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:31.716541    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb5br\" (UniqueName: \"kubernetes.io/projected/af08e083-3649-4feb-8f36-2eb01da9a3db-kube-api-access-kb5br\") pod \"payload-pre-processing-e2e-autores-2c1e69-6cd8cbffb6-k5pwp\" (UID: \"af08e083-3649-4feb-8f36-2eb01da9a3db\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2c1e69-6cd8cbffb6-k5pwp"
Sep 11 01:12:31.716713 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:31.716697    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swbb6\" (UniqueName: \"kubernetes.io/projected/93b6d88b-42f3-44ae-b042-54e5f8e0f763-kube-api-access-swbb6\") pod \"payload-processing-e2e-autores-2c1e69-6ff7b679df-nvthr\" (UID: \"93b6d88b-42f3-44ae-b042-54e5f8e0f763\") " pod="openshift-ingress/payload-processing-e2e-autores-2c1e69-6ff7b679df-nvthr"
Sep 11 01:12:31.757666 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:31.757642    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-2c1e69-6cd8cbffb6-k5pwp"
Sep 11 01:12:31.835120 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:31.835079    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-2c1e69-6ff7b679df-nvthr"
Sep 11 01:12:32.028305 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:32.028282    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2c1e69-6cd8cbffb6-k5pwp"]
Sep 11 01:12:32.030570 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:12:32.030545    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf08e083_3649_4feb_8f36_2eb01da9a3db.slice/crio-81b4f477d78d11198d53ee903fc0434c8981c9469aef335d41227a18f5adc871 WatchSource:0}: Error finding container 81b4f477d78d11198d53ee903fc0434c8981c9469aef335d41227a18f5adc871: Status 404 returned error can't find the container with id 81b4f477d78d11198d53ee903fc0434c8981c9469aef335d41227a18f5adc871
Sep 11 01:12:32.109544 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:32.109470    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-2c1e69-6ff7b679df-nvthr"]
Sep 11 01:12:32.114723 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:12:32.114700    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93b6d88b_42f3_44ae_b042_54e5f8e0f763.slice/crio-b53732f8a86205725a9e3f208203f454d803beb46a7ca572584f779ab652007f WatchSource:0}: Error finding container b53732f8a86205725a9e3f208203f454d803beb46a7ca572584f779ab652007f: Status 404 returned error can't find the container with id b53732f8a86205725a9e3f208203f454d803beb46a7ca572584f779ab652007f
Sep 11 01:12:32.868784 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:32.868753    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-2c1e69-6cd8cbffb6-k5pwp" event={"ID":"af08e083-3649-4feb-8f36-2eb01da9a3db","Type":"ContainerStarted","Data":"6d4d8016a5e811e5b8cae85603c243aa8cf3ef9dec8bc9ecea836bee82193cc3"}
Sep 11 01:12:32.869281 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:32.868807    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-2c1e69-6cd8cbffb6-k5pwp" event={"ID":"af08e083-3649-4feb-8f36-2eb01da9a3db","Type":"ContainerStarted","Data":"81b4f477d78d11198d53ee903fc0434c8981c9469aef335d41227a18f5adc871"}
Sep 11 01:12:32.869281 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:32.868861    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-2c1e69-6cd8cbffb6-k5pwp"
Sep 11 01:12:32.870603 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:32.870573    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-2c1e69-6ff7b679df-nvthr" event={"ID":"93b6d88b-42f3-44ae-b042-54e5f8e0f763","Type":"ContainerStarted","Data":"87133ddf8820decfdc5c1bff4ecd5d6c1044a4d3d974e5d212a504e9b42ee787"}
Sep 11 01:12:32.870739 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:32.870611    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-2c1e69-6ff7b679df-nvthr" event={"ID":"93b6d88b-42f3-44ae-b042-54e5f8e0f763","Type":"ContainerStarted","Data":"b53732f8a86205725a9e3f208203f454d803beb46a7ca572584f779ab652007f"}
Sep 11 01:12:32.870739 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:32.870692    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-2c1e69-6ff7b679df-nvthr"
Sep 11 01:12:32.884502 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:32.884427    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-2c1e69-6cd8cbffb6-k5pwp" podStartSLOduration=1.884401473 podStartE2EDuration="1.884401473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:12:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:12:32.884174154 +0000 UTC m=+868.281366328" watchObservedRunningTime="2026-09-11 01:12:32.884401473 +0000 UTC m=+868.281593648"
Sep 11 01:12:32.900601 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:32.900558    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-2c1e69-6ff7b679df-nvthr" podStartSLOduration=1.900545924 podStartE2EDuration="1.900545924s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:12:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:12:32.899251837 +0000 UTC m=+868.296444010" watchObservedRunningTime="2026-09-11 01:12:32.900545924 +0000 UTC m=+868.297738098"
Sep 11 01:12:33.033330 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:33.033293    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:33.122750 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:33.122674    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:33.817635 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:33.817602    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e98d1e-98bcf4-brhgp"
Sep 11 01:12:34.490476 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:34.490436    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:34.593298 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:34.593260    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:34.821579 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:34.821487    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-e98d1e-65864c5fb5-ggszd"
Sep 11 01:12:34.821579 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:34.821567    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-f4de10-857b967875-n67cq"
Sep 11 01:12:34.832107 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:34.832078    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:35.180194 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:35.180165    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:35.282186 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:35.282153    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:35.282940 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:35.282896    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fd82e780-688877958b-mb2rw"]
Sep 11 01:12:35.286434 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:35.286420    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd82e780-688877958b-mb2rw"
Sep 11 01:12:35.289118 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:35.289095    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-fd82e780\""
Sep 11 01:12:35.289237 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:35.289136    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-fd82e780-dockercfg-dl5xr\""
Sep 11 01:12:35.290153 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:35.290133    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fd82e780-687dbc8bc5-zpj47"]
Sep 11 01:12:35.293829 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:35.293811    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-fd82e780-687dbc8bc5-zpj47"
Sep 11 01:12:35.297821 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:35.297798    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fd82e780-688877958b-mb2rw"]
Sep 11 01:12:35.305584 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:35.305540    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fd82e780-687dbc8bc5-zpj47"]
Sep 11 01:12:35.381330 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:35.381301    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:35.442796 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:35.442701    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d276cca8-a639-4642-957f-ce1a0fdde89d-plugins-config-volume\") pod \"payload-processing-e2e-ait-fd82e780-687dbc8bc5-zpj47\" (UID: \"d276cca8-a639-4642-957f-ce1a0fdde89d\") " pod="openshift-ingress/payload-processing-e2e-ait-fd82e780-687dbc8bc5-zpj47"
Sep 11 01:12:35.442796 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:35.442767    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcc8w\" (UniqueName: \"kubernetes.io/projected/567b5f0f-21f5-4a52-9cce-ef95e3b0c287-kube-api-access-lcc8w\") pod \"payload-pre-processing-e2e-ait-fd82e780-688877958b-mb2rw\" (UID: \"567b5f0f-21f5-4a52-9cce-ef95e3b0c287\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fd82e780-688877958b-mb2rw"
Sep 11 01:12:35.443087 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:35.442797    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9krs\" (UniqueName: \"kubernetes.io/projected/d276cca8-a639-4642-957f-ce1a0fdde89d-kube-api-access-t9krs\") pod \"payload-processing-e2e-ait-fd82e780-687dbc8bc5-zpj47\" (UID: \"d276cca8-a639-4642-957f-ce1a0fdde89d\") " pod="openshift-ingress/payload-processing-e2e-ait-fd82e780-687dbc8bc5-zpj47"
Sep 11 01:12:35.443087 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:35.442951    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/567b5f0f-21f5-4a52-9cce-ef95e3b0c287-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-fd82e780-688877958b-mb2rw\" (UID: \"567b5f0f-21f5-4a52-9cce-ef95e3b0c287\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fd82e780-688877958b-mb2rw"
Sep 11 01:12:35.479313 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:35.479275    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:35.543499 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:35.543448    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/567b5f0f-21f5-4a52-9cce-ef95e3b0c287-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-fd82e780-688877958b-mb2rw\" (UID: \"567b5f0f-21f5-4a52-9cce-ef95e3b0c287\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fd82e780-688877958b-mb2rw"
Sep 11 01:12:35.543963 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:35.543515    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d276cca8-a639-4642-957f-ce1a0fdde89d-plugins-config-volume\") pod \"payload-processing-e2e-ait-fd82e780-687dbc8bc5-zpj47\" (UID: \"d276cca8-a639-4642-957f-ce1a0fdde89d\") " pod="openshift-ingress/payload-processing-e2e-ait-fd82e780-687dbc8bc5-zpj47"
Sep 11 01:12:35.543963 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:35.543543    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcc8w\" (UniqueName: \"kubernetes.io/projected/567b5f0f-21f5-4a52-9cce-ef95e3b0c287-kube-api-access-lcc8w\") pod \"payload-pre-processing-e2e-ait-fd82e780-688877958b-mb2rw\" (UID: \"567b5f0f-21f5-4a52-9cce-ef95e3b0c287\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fd82e780-688877958b-mb2rw"
Sep 11 01:12:35.543963 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:35.543705    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9krs\" (UniqueName: \"kubernetes.io/projected/d276cca8-a639-4642-957f-ce1a0fdde89d-kube-api-access-t9krs\") pod \"payload-processing-e2e-ait-fd82e780-687dbc8bc5-zpj47\" (UID: \"d276cca8-a639-4642-957f-ce1a0fdde89d\") " pod="openshift-ingress/payload-processing-e2e-ait-fd82e780-687dbc8bc5-zpj47"
Sep 11 01:12:35.544459 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:35.544438    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/567b5f0f-21f5-4a52-9cce-ef95e3b0c287-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-fd82e780-688877958b-mb2rw\" (UID: \"567b5f0f-21f5-4a52-9cce-ef95e3b0c287\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fd82e780-688877958b-mb2rw"
Sep 11 01:12:35.544506 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:35.544438    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d276cca8-a639-4642-957f-ce1a0fdde89d-plugins-config-volume\") pod \"payload-processing-e2e-ait-fd82e780-687dbc8bc5-zpj47\" (UID: \"d276cca8-a639-4642-957f-ce1a0fdde89d\") " pod="openshift-ingress/payload-processing-e2e-ait-fd82e780-687dbc8bc5-zpj47"
Sep 11 01:12:35.551522 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:35.551496    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcc8w\" (UniqueName: \"kubernetes.io/projected/567b5f0f-21f5-4a52-9cce-ef95e3b0c287-kube-api-access-lcc8w\") pod \"payload-pre-processing-e2e-ait-fd82e780-688877958b-mb2rw\" (UID: \"567b5f0f-21f5-4a52-9cce-ef95e3b0c287\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fd82e780-688877958b-mb2rw"
Sep 11 01:12:35.551668 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:35.551651    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9krs\" (UniqueName: \"kubernetes.io/projected/d276cca8-a639-4642-957f-ce1a0fdde89d-kube-api-access-t9krs\") pod \"payload-processing-e2e-ait-fd82e780-687dbc8bc5-zpj47\" (UID: \"d276cca8-a639-4642-957f-ce1a0fdde89d\") " pod="openshift-ingress/payload-processing-e2e-ait-fd82e780-687dbc8bc5-zpj47"
Sep 11 01:12:35.575583 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:35.575535    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:35.602923 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:35.602868    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd82e780-688877958b-mb2rw"
Sep 11 01:12:35.610962 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:35.610942    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-fd82e780-687dbc8bc5-zpj47"
Sep 11 01:12:35.677802 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:35.677774    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:35.828519 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:35.828487    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f4de10-977ff8d6-6g49f"
Sep 11 01:12:35.884166 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:35.884138    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fd82e780-688877958b-mb2rw"]
Sep 11 01:12:35.886466 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:12:35.886437    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod567b5f0f_21f5_4a52_9cce_ef95e3b0c287.slice/crio-bff1c66ac15fa3ff8ef41ee14d270aaaf6e3e0d8740ea98e4ac3f7f0dbfce39e WatchSource:0}: Error finding container bff1c66ac15fa3ff8ef41ee14d270aaaf6e3e0d8740ea98e4ac3f7f0dbfce39e: Status 404 returned error can't find the container with id bff1c66ac15fa3ff8ef41ee14d270aaaf6e3e0d8740ea98e4ac3f7f0dbfce39e
Sep 11 01:12:35.895685 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:35.895661    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fd82e780-687dbc8bc5-zpj47"]
Sep 11 01:12:35.900518 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:12:35.900489    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd276cca8_a639_4642_957f_ce1a0fdde89d.slice/crio-740014500aa7eac7c7283ae7626ccc9b96878d0ec9f3ff5c7620735dcabdfac8 WatchSource:0}: Error finding container 740014500aa7eac7c7283ae7626ccc9b96878d0ec9f3ff5c7620735dcabdfac8: Status 404 returned error can't find the container with id 740014500aa7eac7c7283ae7626ccc9b96878d0ec9f3ff5c7620735dcabdfac8
Sep 11 01:12:36.836305 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:36.836275    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd176e-79cc7879f4-tg9wg"
Sep 11 01:12:36.905312 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:36.905272    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-fd82e780-687dbc8bc5-zpj47" event={"ID":"d276cca8-a639-4642-957f-ce1a0fdde89d","Type":"ContainerStarted","Data":"2cf220ddda77666493f94e6df7dd64c266e2dd468c88c10908644d5042517d90"}
Sep 11 01:12:36.905514 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:36.905402    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-fd82e780-687dbc8bc5-zpj47" event={"ID":"d276cca8-a639-4642-957f-ce1a0fdde89d","Type":"ContainerStarted","Data":"740014500aa7eac7c7283ae7626ccc9b96878d0ec9f3ff5c7620735dcabdfac8"}
Sep 11 01:12:36.905581 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:36.905567    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-fd82e780-687dbc8bc5-zpj47"
Sep 11 01:12:36.907133 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:36.907104    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd82e780-688877958b-mb2rw" event={"ID":"567b5f0f-21f5-4a52-9cce-ef95e3b0c287","Type":"ContainerStarted","Data":"80be332c6b10abf4ccd9a4f99f1b0d4454ecc0c3ab5fef8043cb532583a511a2"}
Sep 11 01:12:36.907245 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:36.907150    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd82e780-688877958b-mb2rw" event={"ID":"567b5f0f-21f5-4a52-9cce-ef95e3b0c287","Type":"ContainerStarted","Data":"bff1c66ac15fa3ff8ef41ee14d270aaaf6e3e0d8740ea98e4ac3f7f0dbfce39e"}
Sep 11 01:12:36.907245 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:36.907178    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd82e780-688877958b-mb2rw"
Sep 11 01:12:36.923885 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:36.923427    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-fd82e780-687dbc8bc5-zpj47" podStartSLOduration=1.92340541 podStartE2EDuration="1.92340541s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:12:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:12:36.921455948 +0000 UTC m=+872.318648120" watchObservedRunningTime="2026-09-11 01:12:36.92340541 +0000 UTC m=+872.320597586"
Sep 11 01:12:36.938155 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:36.938096    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd82e780-688877958b-mb2rw" podStartSLOduration=1.9380762969999998 podStartE2EDuration="1.938076297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:12:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:12:36.935549845 +0000 UTC m=+872.332742020" watchObservedRunningTime="2026-09-11 01:12:36.938076297 +0000 UTC m=+872.335268473"
Sep 11 01:12:38.420499 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:38.420461    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:40.152257 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:40.152226    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:42.548205 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:42.548170    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:43.878790 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:43.878761    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-2c1e69-6ff7b679df-nvthr"
Sep 11 01:12:43.879190 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:43.878829    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-2c1e69-6cd8cbffb6-k5pwp"
Sep 11 01:12:44.237810 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:44.237776    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:44.847517 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:44.847484    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:45.203278 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:45.203246    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:45.547237 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:45.547148    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:45.861872 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:45.861784    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:46.072020 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:46.071986    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:46.896706 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:46.896677    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:47.771530 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:47.771497    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:47.913991 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:47.913955    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd82e780-688877958b-mb2rw"
Sep 11 01:12:47.914429 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:47.914035    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-fd82e780-687dbc8bc5-zpj47"
Sep 11 01:12:48.533723 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:48.533689    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:49.098728 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:49.098693    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:49.197770 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:49.197735    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:49.998254 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:49.998220    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:50.100795 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:50.100757    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:50.398845 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:50.398772    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:50.540197 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:50.540162    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:50.687146 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:50.687108    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:50.820231 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:50.820197    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:51.223586 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:51.223558    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:51.320831 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:51.320800    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:51.409000 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:51.408966    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-10a215-6bf5478bc4-z948j"]
Sep 11 01:12:51.412828 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:51.412812    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-10a215-6bf5478bc4-z948j"
Sep 11 01:12:51.415361 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:51.415335    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-10a215-dockercfg-gk9kr\""
Sep 11 01:12:51.415361 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:51.415349    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-10a215\""
Sep 11 01:12:51.426559 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:51.426537    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-10a215-6bf5478bc4-z948j"]
Sep 11 01:12:51.490739 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:51.490715    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42h75\" (UniqueName: \"kubernetes.io/projected/7c5b6724-374f-492d-a3e3-e1ce56905d8b-kube-api-access-42h75\") pod \"payload-pre-processing-e2e-inf-b-10a215-6bf5478bc4-z948j\" (UID: \"7c5b6724-374f-492d-a3e3-e1ce56905d8b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-10a215-6bf5478bc4-z948j"
Sep 11 01:12:51.490847 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:51.490775    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c5b6724-374f-492d-a3e3-e1ce56905d8b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-10a215-6bf5478bc4-z948j\" (UID: \"7c5b6724-374f-492d-a3e3-e1ce56905d8b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-10a215-6bf5478bc4-z948j"
Sep 11 01:12:51.591927 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:51.591885    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42h75\" (UniqueName: \"kubernetes.io/projected/7c5b6724-374f-492d-a3e3-e1ce56905d8b-kube-api-access-42h75\") pod \"payload-pre-processing-e2e-inf-b-10a215-6bf5478bc4-z948j\" (UID: \"7c5b6724-374f-492d-a3e3-e1ce56905d8b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-10a215-6bf5478bc4-z948j"
Sep 11 01:12:51.592097 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:51.591986    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c5b6724-374f-492d-a3e3-e1ce56905d8b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-10a215-6bf5478bc4-z948j\" (UID: \"7c5b6724-374f-492d-a3e3-e1ce56905d8b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-10a215-6bf5478bc4-z948j"
Sep 11 01:12:51.592748 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:51.592728    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c5b6724-374f-492d-a3e3-e1ce56905d8b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-10a215-6bf5478bc4-z948j\" (UID: \"7c5b6724-374f-492d-a3e3-e1ce56905d8b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-10a215-6bf5478bc4-z948j"
Sep 11 01:12:51.599811 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:51.599786    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42h75\" (UniqueName: \"kubernetes.io/projected/7c5b6724-374f-492d-a3e3-e1ce56905d8b-kube-api-access-42h75\") pod \"payload-pre-processing-e2e-inf-b-10a215-6bf5478bc4-z948j\" (UID: \"7c5b6724-374f-492d-a3e3-e1ce56905d8b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-10a215-6bf5478bc4-z948j"
Sep 11 01:12:51.751247 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:51.751169    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-10a215-6bf5478bc4-z948j"
Sep 11 01:12:52.023649 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:52.023603    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-10a215-6bf5478bc4-z948j"]
Sep 11 01:12:52.025811 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:12:52.025784    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c5b6724_374f_492d_a3e3_e1ce56905d8b.slice/crio-0ea1c717682c13954058a2b9a73fa6d007c60e4cb1dcb7ed27b8fa9a00944caf WatchSource:0}: Error finding container 0ea1c717682c13954058a2b9a73fa6d007c60e4cb1dcb7ed27b8fa9a00944caf: Status 404 returned error can't find the container with id 0ea1c717682c13954058a2b9a73fa6d007c60e4cb1dcb7ed27b8fa9a00944caf
Sep 11 01:12:52.262194 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:52.262164    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:52.989518 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:52.989477    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-10a215-6bf5478bc4-z948j" event={"ID":"7c5b6724-374f-492d-a3e3-e1ce56905d8b","Type":"ContainerStarted","Data":"82f908c4cf47c74206563f00e9431f81b2ad29dab9d3fb11b691d497a0d686d2"}
Sep 11 01:12:52.989714 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:52.989543    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-10a215-6bf5478bc4-z948j" event={"ID":"7c5b6724-374f-492d-a3e3-e1ce56905d8b","Type":"ContainerStarted","Data":"0ea1c717682c13954058a2b9a73fa6d007c60e4cb1dcb7ed27b8fa9a00944caf"}
Sep 11 01:12:52.989714 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:52.989658    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-10a215-6bf5478bc4-z948j"
Sep 11 01:12:53.004792 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:53.004745    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-10a215-6bf5478bc4-z948j" podStartSLOduration=2.004730529 podStartE2EDuration="2.004730529s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:12:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:12:53.003346928 +0000 UTC m=+888.400539102" watchObservedRunningTime="2026-09-11 01:12:53.004730529 +0000 UTC m=+888.401922702"
Sep 11 01:12:53.167111 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:53.167070    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:53.836683 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:53.836648    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:53.937037 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:53.937002    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:55.729960 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:55.729931    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:58.483468 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:58.483390    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fd82e780-9r9pm"]
Sep 11 01:12:58.490962 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:58.490940    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fd82e780-9r9pm"
Sep 11 01:12:58.492885 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:58.492859    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fd82e780-9r9pm"]
Sep 11 01:12:58.493267 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:58.493244    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 01:12:58.493377 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:58.493297    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q6gbn\""
Sep 11 01:12:58.548612 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:58.548583    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/830d0c2d-d9d1-4c73-89a9-17f163472708-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fd82e780-9r9pm\" (UID: \"830d0c2d-d9d1-4c73-89a9-17f163472708\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fd82e780-9r9pm"
Sep 11 01:12:58.649429 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:58.649397    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/830d0c2d-d9d1-4c73-89a9-17f163472708-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fd82e780-9r9pm\" (UID: \"830d0c2d-d9d1-4c73-89a9-17f163472708\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fd82e780-9r9pm"
Sep 11 01:12:58.650102 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:58.650082    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/830d0c2d-d9d1-4c73-89a9-17f163472708-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fd82e780-9r9pm\" (UID: \"830d0c2d-d9d1-4c73-89a9-17f163472708\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fd82e780-9r9pm"
Sep 11 01:12:58.802927 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:58.802840    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fd82e780-9r9pm"
Sep 11 01:12:58.815433 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:58.815274    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:58.914835 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:58.914800    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:59.046449 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:59.046418    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:12:59.078033 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:12:59.078008    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fd82e780-9r9pm"]
Sep 11 01:12:59.080196 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:12:59.080174    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod830d0c2d_d9d1_4c73_89a9_17f163472708.slice/crio-536c3ff917b72c660cd0e118abd67bf31ace60be51691fee92615856fb19862f WatchSource:0}: Error finding container 536c3ff917b72c660cd0e118abd67bf31ace60be51691fee92615856fb19862f: Status 404 returned error can't find the container with id 536c3ff917b72c660cd0e118abd67bf31ace60be51691fee92615856fb19862f
Sep 11 01:13:00.027438 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:00.027390    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fd82e780-9r9pm" event={"ID":"830d0c2d-d9d1-4c73-89a9-17f163472708","Type":"ContainerStarted","Data":"536c3ff917b72c660cd0e118abd67bf31ace60be51691fee92615856fb19862f"}
Sep 11 01:13:02.044198 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:02.044164    2574 generic.go:356] "Generic (PLEG): container finished" podID="830d0c2d-d9d1-4c73-89a9-17f163472708" containerID="70d1cc586ab9affe3a53d162be33b75356a6da3dc0697f53fdda080a1a726d4b" exitCode=0
Sep 11 01:13:02.044608 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:02.044260    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fd82e780-9r9pm" event={"ID":"830d0c2d-d9d1-4c73-89a9-17f163472708","Type":"ContainerDied","Data":"70d1cc586ab9affe3a53d162be33b75356a6da3dc0697f53fdda080a1a726d4b"}
Sep 11 01:13:03.185992 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:03.185963    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fd82e780-9r9pm"
Sep 11 01:13:03.294115 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:03.294082    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/830d0c2d-d9d1-4c73-89a9-17f163472708-maas-api-service-ca\") pod \"830d0c2d-d9d1-4c73-89a9-17f163472708\" (UID: \"830d0c2d-d9d1-4c73-89a9-17f163472708\") "
Sep 11 01:13:03.294440 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:03.294416    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/830d0c2d-d9d1-4c73-89a9-17f163472708-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "830d0c2d-d9d1-4c73-89a9-17f163472708" (UID: "830d0c2d-d9d1-4c73-89a9-17f163472708"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:13:03.394891 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:03.394812    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/830d0c2d-d9d1-4c73-89a9-17f163472708-maas-api-service-ca\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:13:03.996976 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:03.996943    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-10a215-6bf5478bc4-z948j"
Sep 11 01:13:04.057973 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:04.057938    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fd82e780-9r9pm" event={"ID":"830d0c2d-d9d1-4c73-89a9-17f163472708","Type":"ContainerDied","Data":"536c3ff917b72c660cd0e118abd67bf31ace60be51691fee92615856fb19862f"}
Sep 11 01:13:04.057973 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:04.057970    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fd82e780-9r9pm"
Sep 11 01:13:04.058176 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:04.057999    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="536c3ff917b72c660cd0e118abd67bf31ace60be51691fee92615856fb19862f"
Sep 11 01:13:04.934317 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:04.934280    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:05.038235 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:05.038197    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:05.139463 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:05.139431    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:05.140201 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:05.140173    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fd82e780-687dbc8bc5-zpj47"]
Sep 11 01:13:05.140452 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:05.140429    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-fd82e780-687dbc8bc5-zpj47" podUID="d276cca8-a639-4642-957f-ce1a0fdde89d" containerName="payload-processing" containerID="cri-o://2cf220ddda77666493f94e6df7dd64c266e2dd468c88c10908644d5042517d90" gracePeriod=30
Sep 11 01:13:05.146184 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:05.146147    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fd82e780-688877958b-mb2rw"]
Sep 11 01:13:05.146407 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:05.146360    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd82e780-688877958b-mb2rw" podUID="567b5f0f-21f5-4a52-9cce-ef95e3b0c287" containerName="payload-pre-processing" containerID="cri-o://80be332c6b10abf4ccd9a4f99f1b0d4454ecc0c3ab5fef8043cb532583a511a2" gracePeriod=30
Sep 11 01:13:05.237391 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:05.237365    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:05.338300 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:05.338273    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:05.378399 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:05.378371    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fd82e780-9r9pm"]
Sep 11 01:13:05.380507 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:05.380483    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fd82e780-9r9pm"]
Sep 11 01:13:05.393334 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:05.393317    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-fd82e780-687dbc8bc5-zpj47"
Sep 11 01:13:05.424240 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:05.424176    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:05.450360 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:05.450339    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd82e780-688877958b-mb2rw"
Sep 11 01:13:05.523007 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:05.522925    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/567b5f0f-21f5-4a52-9cce-ef95e3b0c287-plugins-config-volume\") pod \"567b5f0f-21f5-4a52-9cce-ef95e3b0c287\" (UID: \"567b5f0f-21f5-4a52-9cce-ef95e3b0c287\") "
Sep 11 01:13:05.523170 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:05.523031    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lcc8w\" (UniqueName: \"kubernetes.io/projected/567b5f0f-21f5-4a52-9cce-ef95e3b0c287-kube-api-access-lcc8w\") pod \"567b5f0f-21f5-4a52-9cce-ef95e3b0c287\" (UID: \"567b5f0f-21f5-4a52-9cce-ef95e3b0c287\") "
Sep 11 01:13:05.523170 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:05.523077    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d276cca8-a639-4642-957f-ce1a0fdde89d-plugins-config-volume\") pod \"d276cca8-a639-4642-957f-ce1a0fdde89d\" (UID: \"d276cca8-a639-4642-957f-ce1a0fdde89d\") "
Sep 11 01:13:05.523170 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:05.523097    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9krs\" (UniqueName: \"kubernetes.io/projected/d276cca8-a639-4642-957f-ce1a0fdde89d-kube-api-access-t9krs\") pod \"d276cca8-a639-4642-957f-ce1a0fdde89d\" (UID: \"d276cca8-a639-4642-957f-ce1a0fdde89d\") "
Sep 11 01:13:05.523402 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:05.523372    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/567b5f0f-21f5-4a52-9cce-ef95e3b0c287-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "567b5f0f-21f5-4a52-9cce-ef95e3b0c287" (UID: "567b5f0f-21f5-4a52-9cce-ef95e3b0c287"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:13:05.523466 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:05.523448    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d276cca8-a639-4642-957f-ce1a0fdde89d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d276cca8-a639-4642-957f-ce1a0fdde89d" (UID: "d276cca8-a639-4642-957f-ce1a0fdde89d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:13:05.523680 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:05.523654    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:05.525389 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:05.525368    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/567b5f0f-21f5-4a52-9cce-ef95e3b0c287-kube-api-access-lcc8w" (OuterVolumeSpecName: "kube-api-access-lcc8w") pod "567b5f0f-21f5-4a52-9cce-ef95e3b0c287" (UID: "567b5f0f-21f5-4a52-9cce-ef95e3b0c287"). InnerVolumeSpecName "kube-api-access-lcc8w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:13:05.525468 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:05.525420    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d276cca8-a639-4642-957f-ce1a0fdde89d-kube-api-access-t9krs" (OuterVolumeSpecName: "kube-api-access-t9krs") pod "d276cca8-a639-4642-957f-ce1a0fdde89d" (UID: "d276cca8-a639-4642-957f-ce1a0fdde89d"). InnerVolumeSpecName "kube-api-access-t9krs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:13:05.624393 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:05.624357    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/567b5f0f-21f5-4a52-9cce-ef95e3b0c287-plugins-config-volume\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:13:05.624393 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:05.624387    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lcc8w\" (UniqueName: \"kubernetes.io/projected/567b5f0f-21f5-4a52-9cce-ef95e3b0c287-kube-api-access-lcc8w\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:13:05.624582 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:05.624401    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d276cca8-a639-4642-957f-ce1a0fdde89d-plugins-config-volume\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:13:05.624582 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:05.624413    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9krs\" (UniqueName: \"kubernetes.io/projected/d276cca8-a639-4642-957f-ce1a0fdde89d-kube-api-access-t9krs\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:13:06.007295 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:06.007264    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:06.071172 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:06.071139    2574 generic.go:356] "Generic (PLEG): container finished" podID="567b5f0f-21f5-4a52-9cce-ef95e3b0c287" containerID="80be332c6b10abf4ccd9a4f99f1b0d4454ecc0c3ab5fef8043cb532583a511a2" exitCode=0
Sep 11 01:13:06.071365 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:06.071191    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd82e780-688877958b-mb2rw"
Sep 11 01:13:06.071365 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:06.071219    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd82e780-688877958b-mb2rw" event={"ID":"567b5f0f-21f5-4a52-9cce-ef95e3b0c287","Type":"ContainerDied","Data":"80be332c6b10abf4ccd9a4f99f1b0d4454ecc0c3ab5fef8043cb532583a511a2"}
Sep 11 01:13:06.071365 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:06.071270    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd82e780-688877958b-mb2rw" event={"ID":"567b5f0f-21f5-4a52-9cce-ef95e3b0c287","Type":"ContainerDied","Data":"bff1c66ac15fa3ff8ef41ee14d270aaaf6e3e0d8740ea98e4ac3f7f0dbfce39e"}
Sep 11 01:13:06.071365 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:06.071295    2574 scope.go:117] "RemoveContainer" containerID="80be332c6b10abf4ccd9a4f99f1b0d4454ecc0c3ab5fef8043cb532583a511a2"
Sep 11 01:13:06.072711 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:06.072683    2574 generic.go:356] "Generic (PLEG): container finished" podID="d276cca8-a639-4642-957f-ce1a0fdde89d" containerID="2cf220ddda77666493f94e6df7dd64c266e2dd468c88c10908644d5042517d90" exitCode=0
Sep 11 01:13:06.072832 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:06.072717    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-fd82e780-687dbc8bc5-zpj47" event={"ID":"d276cca8-a639-4642-957f-ce1a0fdde89d","Type":"ContainerDied","Data":"2cf220ddda77666493f94e6df7dd64c266e2dd468c88c10908644d5042517d90"}
Sep 11 01:13:06.072832 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:06.072732    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-fd82e780-687dbc8bc5-zpj47"
Sep 11 01:13:06.072832 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:06.072780    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-fd82e780-687dbc8bc5-zpj47" event={"ID":"d276cca8-a639-4642-957f-ce1a0fdde89d","Type":"ContainerDied","Data":"740014500aa7eac7c7283ae7626ccc9b96878d0ec9f3ff5c7620735dcabdfac8"}
Sep 11 01:13:06.082109 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:06.081903    2574 scope.go:117] "RemoveContainer" containerID="80be332c6b10abf4ccd9a4f99f1b0d4454ecc0c3ab5fef8043cb532583a511a2"
Sep 11 01:13:06.082254 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:13:06.082231    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80be332c6b10abf4ccd9a4f99f1b0d4454ecc0c3ab5fef8043cb532583a511a2\": container with ID starting with 80be332c6b10abf4ccd9a4f99f1b0d4454ecc0c3ab5fef8043cb532583a511a2 not found: ID does not exist" containerID="80be332c6b10abf4ccd9a4f99f1b0d4454ecc0c3ab5fef8043cb532583a511a2"
Sep 11 01:13:06.082319 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:06.082265    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80be332c6b10abf4ccd9a4f99f1b0d4454ecc0c3ab5fef8043cb532583a511a2"} err="failed to get container status \"80be332c6b10abf4ccd9a4f99f1b0d4454ecc0c3ab5fef8043cb532583a511a2\": rpc error: code = NotFound desc = could not find container \"80be332c6b10abf4ccd9a4f99f1b0d4454ecc0c3ab5fef8043cb532583a511a2\": container with ID starting with 80be332c6b10abf4ccd9a4f99f1b0d4454ecc0c3ab5fef8043cb532583a511a2 not found: ID does not exist"
Sep 11 01:13:06.082319 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:06.082282    2574 scope.go:117] "RemoveContainer" containerID="2cf220ddda77666493f94e6df7dd64c266e2dd468c88c10908644d5042517d90"
Sep 11 01:13:06.092040 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:06.092021    2574 scope.go:117] "RemoveContainer" containerID="2cf220ddda77666493f94e6df7dd64c266e2dd468c88c10908644d5042517d90"
Sep 11 01:13:06.092721 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:13:06.092701    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cf220ddda77666493f94e6df7dd64c266e2dd468c88c10908644d5042517d90\": container with ID starting with 2cf220ddda77666493f94e6df7dd64c266e2dd468c88c10908644d5042517d90 not found: ID does not exist" containerID="2cf220ddda77666493f94e6df7dd64c266e2dd468c88c10908644d5042517d90"
Sep 11 01:13:06.092801 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:06.092726    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cf220ddda77666493f94e6df7dd64c266e2dd468c88c10908644d5042517d90"} err="failed to get container status \"2cf220ddda77666493f94e6df7dd64c266e2dd468c88c10908644d5042517d90\": rpc error: code = NotFound desc = could not find container \"2cf220ddda77666493f94e6df7dd64c266e2dd468c88c10908644d5042517d90\": container with ID starting with 2cf220ddda77666493f94e6df7dd64c266e2dd468c88c10908644d5042517d90 not found: ID does not exist"
Sep 11 01:13:06.097829 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:06.097806    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fd82e780-687dbc8bc5-zpj47"]
Sep 11 01:13:06.100667 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:06.100642    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:06.101755 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:06.101736    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fd82e780-687dbc8bc5-zpj47"]
Sep 11 01:13:06.110835 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:06.110817    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fd82e780-688877958b-mb2rw"]
Sep 11 01:13:06.118788 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:06.118769    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fd82e780-688877958b-mb2rw"]
Sep 11 01:13:06.274600 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:06.274515    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:06.615385 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:06.615293    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:06.868707 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:06.868631    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:07.105931 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:07.105874    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="567b5f0f-21f5-4a52-9cce-ef95e3b0c287" path="/var/lib/kubelet/pods/567b5f0f-21f5-4a52-9cce-ef95e3b0c287/volumes"
Sep 11 01:13:07.106381 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:07.106358    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="830d0c2d-d9d1-4c73-89a9-17f163472708" path="/var/lib/kubelet/pods/830d0c2d-d9d1-4c73-89a9-17f163472708/volumes"
Sep 11 01:13:07.106754 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:07.106740    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d276cca8-a639-4642-957f-ce1a0fdde89d" path="/var/lib/kubelet/pods/d276cca8-a639-4642-957f-ce1a0fdde89d/volumes"
Sep 11 01:13:08.861780 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:08.861728    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:08.918349 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:08.918319    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:09.100251 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:09.100198    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:09.204787 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:09.204757    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:09.385551 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:09.385518    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:09.612466 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:09.612385    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:09.708301 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:09.708270    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:09.712959 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:09.712931    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-0b6ae428-7f54bc76c4-s27dn"]
Sep 11 01:13:09.713363 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:09.713347    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="830d0c2d-d9d1-4c73-89a9-17f163472708" containerName="revoke-keys"
Sep 11 01:13:09.713434 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:09.713366    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="830d0c2d-d9d1-4c73-89a9-17f163472708" containerName="revoke-keys"
Sep 11 01:13:09.713434 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:09.713387    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d276cca8-a639-4642-957f-ce1a0fdde89d" containerName="payload-processing"
Sep 11 01:13:09.713434 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:09.713396    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d276cca8-a639-4642-957f-ce1a0fdde89d" containerName="payload-processing"
Sep 11 01:13:09.713434 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:09.713409    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="567b5f0f-21f5-4a52-9cce-ef95e3b0c287" containerName="payload-pre-processing"
Sep 11 01:13:09.713434 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:09.713418    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="567b5f0f-21f5-4a52-9cce-ef95e3b0c287" containerName="payload-pre-processing"
Sep 11 01:13:09.713685 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:09.713514    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="830d0c2d-d9d1-4c73-89a9-17f163472708" containerName="revoke-keys"
Sep 11 01:13:09.713685 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:09.713529    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="567b5f0f-21f5-4a52-9cce-ef95e3b0c287" containerName="payload-pre-processing"
Sep 11 01:13:09.713685 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:09.713538    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d276cca8-a639-4642-957f-ce1a0fdde89d" containerName="payload-processing"
Sep 11 01:13:09.718181 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:09.718160    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0b6ae428-7f54bc76c4-s27dn"
Sep 11 01:13:09.720289 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:09.720272    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-0b6ae428\""
Sep 11 01:13:09.721137 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:09.721120    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-0b6ae428-dockercfg-xm6rd\""
Sep 11 01:13:09.728112 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:09.728090    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-0b6ae428-7f54bc76c4-s27dn"]
Sep 11 01:13:09.758239 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:09.758213    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0b6ae428-d9c99cfd-pgz6v"]
Sep 11 01:13:09.761621 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:09.761606    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-0b6ae428-d9c99cfd-pgz6v"
Sep 11 01:13:09.770795 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:09.770773    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0b6ae428-d9c99cfd-pgz6v"]
Sep 11 01:13:09.860742 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:09.860708    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlfhp\" (UniqueName: \"kubernetes.io/projected/d903fb9a-cd96-4c42-9dc8-6e4069e4eb58-kube-api-access-xlfhp\") pod \"payload-pre-processing-e2e-migrate-0b6ae428-7f54bc76c4-s27dn\" (UID: \"d903fb9a-cd96-4c42-9dc8-6e4069e4eb58\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-0b6ae428-7f54bc76c4-s27dn"
Sep 11 01:13:09.860942 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:09.860758    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e83ff0c-f75d-47ce-915e-cb475b4a0394-plugins-config-volume\") pod \"payload-processing-e2e-migrate-0b6ae428-d9c99cfd-pgz6v\" (UID: \"3e83ff0c-f75d-47ce-915e-cb475b4a0394\") " pod="openshift-ingress/payload-processing-e2e-migrate-0b6ae428-d9c99cfd-pgz6v"
Sep 11 01:13:09.860942 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:09.860841    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d903fb9a-cd96-4c42-9dc8-6e4069e4eb58-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-0b6ae428-7f54bc76c4-s27dn\" (UID: \"d903fb9a-cd96-4c42-9dc8-6e4069e4eb58\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-0b6ae428-7f54bc76c4-s27dn"
Sep 11 01:13:09.861050 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:09.860954    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4szg\" (UniqueName: \"kubernetes.io/projected/3e83ff0c-f75d-47ce-915e-cb475b4a0394-kube-api-access-j4szg\") pod \"payload-processing-e2e-migrate-0b6ae428-d9c99cfd-pgz6v\" (UID: \"3e83ff0c-f75d-47ce-915e-cb475b4a0394\") " pod="openshift-ingress/payload-processing-e2e-migrate-0b6ae428-d9c99cfd-pgz6v"
Sep 11 01:13:09.962252 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:09.962206    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4szg\" (UniqueName: \"kubernetes.io/projected/3e83ff0c-f75d-47ce-915e-cb475b4a0394-kube-api-access-j4szg\") pod \"payload-processing-e2e-migrate-0b6ae428-d9c99cfd-pgz6v\" (UID: \"3e83ff0c-f75d-47ce-915e-cb475b4a0394\") " pod="openshift-ingress/payload-processing-e2e-migrate-0b6ae428-d9c99cfd-pgz6v"
Sep 11 01:13:09.962730 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:09.962288    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlfhp\" (UniqueName: \"kubernetes.io/projected/d903fb9a-cd96-4c42-9dc8-6e4069e4eb58-kube-api-access-xlfhp\") pod \"payload-pre-processing-e2e-migrate-0b6ae428-7f54bc76c4-s27dn\" (UID: \"d903fb9a-cd96-4c42-9dc8-6e4069e4eb58\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-0b6ae428-7f54bc76c4-s27dn"
Sep 11 01:13:09.962730 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:09.962317    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e83ff0c-f75d-47ce-915e-cb475b4a0394-plugins-config-volume\") pod \"payload-processing-e2e-migrate-0b6ae428-d9c99cfd-pgz6v\" (UID: \"3e83ff0c-f75d-47ce-915e-cb475b4a0394\") " pod="openshift-ingress/payload-processing-e2e-migrate-0b6ae428-d9c99cfd-pgz6v"
Sep 11 01:13:09.962730 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:09.962342    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d903fb9a-cd96-4c42-9dc8-6e4069e4eb58-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-0b6ae428-7f54bc76c4-s27dn\" (UID: \"d903fb9a-cd96-4c42-9dc8-6e4069e4eb58\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-0b6ae428-7f54bc76c4-s27dn"
Sep 11 01:13:09.963030 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:09.963010    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d903fb9a-cd96-4c42-9dc8-6e4069e4eb58-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-0b6ae428-7f54bc76c4-s27dn\" (UID: \"d903fb9a-cd96-4c42-9dc8-6e4069e4eb58\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-0b6ae428-7f54bc76c4-s27dn"
Sep 11 01:13:09.963168 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:09.963146    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e83ff0c-f75d-47ce-915e-cb475b4a0394-plugins-config-volume\") pod \"payload-processing-e2e-migrate-0b6ae428-d9c99cfd-pgz6v\" (UID: \"3e83ff0c-f75d-47ce-915e-cb475b4a0394\") " pod="openshift-ingress/payload-processing-e2e-migrate-0b6ae428-d9c99cfd-pgz6v"
Sep 11 01:13:09.970006 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:09.969981    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4szg\" (UniqueName: \"kubernetes.io/projected/3e83ff0c-f75d-47ce-915e-cb475b4a0394-kube-api-access-j4szg\") pod \"payload-processing-e2e-migrate-0b6ae428-d9c99cfd-pgz6v\" (UID: \"3e83ff0c-f75d-47ce-915e-cb475b4a0394\") " pod="openshift-ingress/payload-processing-e2e-migrate-0b6ae428-d9c99cfd-pgz6v"
Sep 11 01:13:09.970170 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:09.970150    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlfhp\" (UniqueName: \"kubernetes.io/projected/d903fb9a-cd96-4c42-9dc8-6e4069e4eb58-kube-api-access-xlfhp\") pod \"payload-pre-processing-e2e-migrate-0b6ae428-7f54bc76c4-s27dn\" (UID: \"d903fb9a-cd96-4c42-9dc8-6e4069e4eb58\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-0b6ae428-7f54bc76c4-s27dn"
Sep 11 01:13:10.029585 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:10.029559    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0b6ae428-7f54bc76c4-s27dn"
Sep 11 01:13:10.072820 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:10.072787    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-0b6ae428-d9c99cfd-pgz6v"
Sep 11 01:13:10.302330 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:10.302305    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-0b6ae428-7f54bc76c4-s27dn"]
Sep 11 01:13:10.304276 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:13:10.304251    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd903fb9a_cd96_4c42_9dc8_6e4069e4eb58.slice/crio-fd9d38c645b9fcf004d5e6e011c750fc66566fbb2078508524e303eda1a90a1a WatchSource:0}: Error finding container fd9d38c645b9fcf004d5e6e011c750fc66566fbb2078508524e303eda1a90a1a: Status 404 returned error can't find the container with id fd9d38c645b9fcf004d5e6e011c750fc66566fbb2078508524e303eda1a90a1a
Sep 11 01:13:10.352476 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:10.352450    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0b6ae428-d9c99cfd-pgz6v"]
Sep 11 01:13:10.355927 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:13:10.355882    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e83ff0c_f75d_47ce_915e_cb475b4a0394.slice/crio-84df1215cc449242dbd81db2359d6b2dd5c92df95b8acdad02789f342bba4fa0 WatchSource:0}: Error finding container 84df1215cc449242dbd81db2359d6b2dd5c92df95b8acdad02789f342bba4fa0: Status 404 returned error can't find the container with id 84df1215cc449242dbd81db2359d6b2dd5c92df95b8acdad02789f342bba4fa0
Sep 11 01:13:11.113191 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:11.113153    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0b6ae428-7f54bc76c4-s27dn" event={"ID":"d903fb9a-cd96-4c42-9dc8-6e4069e4eb58","Type":"ContainerStarted","Data":"07ce00f89f35e5a44ac66371fafb1d2ba3df21eebd1ac3e5357f4f2e0421e787"}
Sep 11 01:13:11.113670 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:11.113215    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0b6ae428-7f54bc76c4-s27dn" event={"ID":"d903fb9a-cd96-4c42-9dc8-6e4069e4eb58","Type":"ContainerStarted","Data":"fd9d38c645b9fcf004d5e6e011c750fc66566fbb2078508524e303eda1a90a1a"}
Sep 11 01:13:11.113670 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:11.113247    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0b6ae428-7f54bc76c4-s27dn"
Sep 11 01:13:11.114896 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:11.114869    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-0b6ae428-d9c99cfd-pgz6v" event={"ID":"3e83ff0c-f75d-47ce-915e-cb475b4a0394","Type":"ContainerStarted","Data":"9042bd6d5eca969e07571cf8efe0f8d2efb7ccc9ce55971e2a5ce80d6f63424c"}
Sep 11 01:13:11.115016 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:11.114933    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-0b6ae428-d9c99cfd-pgz6v" event={"ID":"3e83ff0c-f75d-47ce-915e-cb475b4a0394","Type":"ContainerStarted","Data":"84df1215cc449242dbd81db2359d6b2dd5c92df95b8acdad02789f342bba4fa0"}
Sep 11 01:13:11.115016 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:11.114992    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-0b6ae428-d9c99cfd-pgz6v"
Sep 11 01:13:11.138474 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:11.138420    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0b6ae428-7f54bc76c4-s27dn" podStartSLOduration=2.138406134 podStartE2EDuration="2.138406134s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:13:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:13:11.136357957 +0000 UTC m=+906.533550155" watchObservedRunningTime="2026-09-11 01:13:11.138406134 +0000 UTC m=+906.535598307"
Sep 11 01:13:11.159007 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:11.158962    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-0b6ae428-d9c99cfd-pgz6v" podStartSLOduration=2.158946833 podStartE2EDuration="2.158946833s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:13:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:13:11.157123705 +0000 UTC m=+906.554315878" watchObservedRunningTime="2026-09-11 01:13:11.158946833 +0000 UTC m=+906.556139006"
Sep 11 01:13:11.644286 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:11.644251    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:11.744092 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:11.744061    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:14.148396 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:14.148360    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:18.857798 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:18.857764    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:18.958083 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:18.958040    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:22.122099 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:22.122069    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-0b6ae428-d9c99cfd-pgz6v"
Sep 11 01:13:22.122567 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:22.122144    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0b6ae428-7f54bc76c4-s27dn"
Sep 11 01:13:23.495587 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:23.495552    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:30.051307 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:30.051268    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:33.922492 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:33.922461    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b6ae428-8pj2g"]
Sep 11 01:13:33.926847 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:33.926828    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b6ae428-8pj2g"
Sep 11 01:13:33.929132 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:33.929109    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 01:13:33.929245 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:33.929109    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q6gbn\""
Sep 11 01:13:33.936079 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:33.936055    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b6ae428-8pj2g"]
Sep 11 01:13:33.961804 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:33.961770    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/927405cc-d4c0-48d9-82ac-8637fd369398-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-0b6ae428-8pj2g\" (UID: \"927405cc-d4c0-48d9-82ac-8637fd369398\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b6ae428-8pj2g"
Sep 11 01:13:34.062709 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:34.062670    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/927405cc-d4c0-48d9-82ac-8637fd369398-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-0b6ae428-8pj2g\" (UID: \"927405cc-d4c0-48d9-82ac-8637fd369398\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b6ae428-8pj2g"
Sep 11 01:13:34.063287 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:34.063269    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/927405cc-d4c0-48d9-82ac-8637fd369398-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-0b6ae428-8pj2g\" (UID: \"927405cc-d4c0-48d9-82ac-8637fd369398\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b6ae428-8pj2g"
Sep 11 01:13:34.241638 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:34.241551    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b6ae428-8pj2g"
Sep 11 01:13:34.511715 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:34.511686    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b6ae428-8pj2g"]
Sep 11 01:13:34.513655 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:13:34.513618    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod927405cc_d4c0_48d9_82ac_8637fd369398.slice/crio-5f603f55c484949f3e48e0d1b3ad003e4e369d0b8d173ea2eb5f1a2647044e36 WatchSource:0}: Error finding container 5f603f55c484949f3e48e0d1b3ad003e4e369d0b8d173ea2eb5f1a2647044e36: Status 404 returned error can't find the container with id 5f603f55c484949f3e48e0d1b3ad003e4e369d0b8d173ea2eb5f1a2647044e36
Sep 11 01:13:35.243785 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:35.243751    2574 generic.go:356] "Generic (PLEG): container finished" podID="927405cc-d4c0-48d9-82ac-8637fd369398" containerID="9f25c1aa6cc2c9f89ca7080b5a4966a38ad9dd21836bd0ce36b174cc7c161226" exitCode=0
Sep 11 01:13:35.244247 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:35.243838    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b6ae428-8pj2g" event={"ID":"927405cc-d4c0-48d9-82ac-8637fd369398","Type":"ContainerDied","Data":"9f25c1aa6cc2c9f89ca7080b5a4966a38ad9dd21836bd0ce36b174cc7c161226"}
Sep 11 01:13:35.244247 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:35.243895    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b6ae428-8pj2g" event={"ID":"927405cc-d4c0-48d9-82ac-8637fd369398","Type":"ContainerStarted","Data":"5f603f55c484949f3e48e0d1b3ad003e4e369d0b8d173ea2eb5f1a2647044e36"}
Sep 11 01:13:36.386881 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:36.386861    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b6ae428-8pj2g"
Sep 11 01:13:36.484082 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:36.484052    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/927405cc-d4c0-48d9-82ac-8637fd369398-maas-api-service-ca\") pod \"927405cc-d4c0-48d9-82ac-8637fd369398\" (UID: \"927405cc-d4c0-48d9-82ac-8637fd369398\") "
Sep 11 01:13:36.484372 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:36.484348    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/927405cc-d4c0-48d9-82ac-8637fd369398-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "927405cc-d4c0-48d9-82ac-8637fd369398" (UID: "927405cc-d4c0-48d9-82ac-8637fd369398"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:13:36.585536 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:36.585477    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/927405cc-d4c0-48d9-82ac-8637fd369398-maas-api-service-ca\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:13:37.255355 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:37.255326    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b6ae428-8pj2g"
Sep 11 01:13:37.255355 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:37.255337    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b6ae428-8pj2g" event={"ID":"927405cc-d4c0-48d9-82ac-8637fd369398","Type":"ContainerDied","Data":"5f603f55c484949f3e48e0d1b3ad003e4e369d0b8d173ea2eb5f1a2647044e36"}
Sep 11 01:13:37.255567 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:37.255378    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5f603f55c484949f3e48e0d1b3ad003e4e369d0b8d173ea2eb5f1a2647044e36"
Sep 11 01:13:40.487745 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:40.487703    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:40.504110 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:40.504080    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0b6ae428-d9c99cfd-pgz6v"]
Sep 11 01:13:40.504346 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:40.504321    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-0b6ae428-d9c99cfd-pgz6v" podUID="3e83ff0c-f75d-47ce-915e-cb475b4a0394" containerName="payload-processing" containerID="cri-o://9042bd6d5eca969e07571cf8efe0f8d2efb7ccc9ce55971e2a5ce80d6f63424c" gracePeriod=30
Sep 11 01:13:40.510326 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:40.509721    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-0b6ae428-7f54bc76c4-s27dn"]
Sep 11 01:13:40.510326 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:40.509972    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0b6ae428-7f54bc76c4-s27dn" podUID="d903fb9a-cd96-4c42-9dc8-6e4069e4eb58" containerName="payload-pre-processing" containerID="cri-o://07ce00f89f35e5a44ac66371fafb1d2ba3df21eebd1ac3e5357f4f2e0421e787" gracePeriod=30
Sep 11 01:13:40.588874 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:40.588849    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:40.685503 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:40.685473    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:40.691415 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:40.691389    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b6ae428-8pj2g"]
Sep 11 01:13:40.696861 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:40.696837    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0b6ae428-8pj2g"]
Sep 11 01:13:40.781744 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:40.781723    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-0b6ae428-d9c99cfd-pgz6v"
Sep 11 01:13:40.784665 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:40.784635    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0b6ae428-7f54bc76c4-s27dn"
Sep 11 01:13:40.785067 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:40.785045    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:40.820111 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:40.820077    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlfhp\" (UniqueName: \"kubernetes.io/projected/d903fb9a-cd96-4c42-9dc8-6e4069e4eb58-kube-api-access-xlfhp\") pod \"d903fb9a-cd96-4c42-9dc8-6e4069e4eb58\" (UID: \"d903fb9a-cd96-4c42-9dc8-6e4069e4eb58\") "
Sep 11 01:13:40.820258 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:40.820152    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j4szg\" (UniqueName: \"kubernetes.io/projected/3e83ff0c-f75d-47ce-915e-cb475b4a0394-kube-api-access-j4szg\") pod \"3e83ff0c-f75d-47ce-915e-cb475b4a0394\" (UID: \"3e83ff0c-f75d-47ce-915e-cb475b4a0394\") "
Sep 11 01:13:40.820258 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:40.820199    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d903fb9a-cd96-4c42-9dc8-6e4069e4eb58-plugins-config-volume\") pod \"d903fb9a-cd96-4c42-9dc8-6e4069e4eb58\" (UID: \"d903fb9a-cd96-4c42-9dc8-6e4069e4eb58\") "
Sep 11 01:13:40.820258 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:40.820243    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e83ff0c-f75d-47ce-915e-cb475b4a0394-plugins-config-volume\") pod \"3e83ff0c-f75d-47ce-915e-cb475b4a0394\" (UID: \"3e83ff0c-f75d-47ce-915e-cb475b4a0394\") "
Sep 11 01:13:40.821211 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:40.821170    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d903fb9a-cd96-4c42-9dc8-6e4069e4eb58-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d903fb9a-cd96-4c42-9dc8-6e4069e4eb58" (UID: "d903fb9a-cd96-4c42-9dc8-6e4069e4eb58"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:13:40.821327 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:40.821203    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e83ff0c-f75d-47ce-915e-cb475b4a0394-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3e83ff0c-f75d-47ce-915e-cb475b4a0394" (UID: "3e83ff0c-f75d-47ce-915e-cb475b4a0394"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:13:40.824311 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:40.824278    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e83ff0c-f75d-47ce-915e-cb475b4a0394-kube-api-access-j4szg" (OuterVolumeSpecName: "kube-api-access-j4szg") pod "3e83ff0c-f75d-47ce-915e-cb475b4a0394" (UID: "3e83ff0c-f75d-47ce-915e-cb475b4a0394"). InnerVolumeSpecName "kube-api-access-j4szg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:13:40.824439 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:40.824418    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d903fb9a-cd96-4c42-9dc8-6e4069e4eb58-kube-api-access-xlfhp" (OuterVolumeSpecName: "kube-api-access-xlfhp") pod "d903fb9a-cd96-4c42-9dc8-6e4069e4eb58" (UID: "d903fb9a-cd96-4c42-9dc8-6e4069e4eb58"). InnerVolumeSpecName "kube-api-access-xlfhp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:13:40.871667 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:40.871634    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:40.921970 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:40.921933    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlfhp\" (UniqueName: \"kubernetes.io/projected/d903fb9a-cd96-4c42-9dc8-6e4069e4eb58-kube-api-access-xlfhp\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:13:40.921970 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:40.921965    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4szg\" (UniqueName: \"kubernetes.io/projected/3e83ff0c-f75d-47ce-915e-cb475b4a0394-kube-api-access-j4szg\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:13:40.921970 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:40.921977    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d903fb9a-cd96-4c42-9dc8-6e4069e4eb58-plugins-config-volume\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:13:40.922182 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:40.921989    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e83ff0c-f75d-47ce-915e-cb475b4a0394-plugins-config-volume\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:13:41.100298 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:41.100221    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="927405cc-d4c0-48d9-82ac-8637fd369398" path="/var/lib/kubelet/pods/927405cc-d4c0-48d9-82ac-8637fd369398/volumes"
Sep 11 01:13:41.275649 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:41.275616    2574 generic.go:356] "Generic (PLEG): container finished" podID="d903fb9a-cd96-4c42-9dc8-6e4069e4eb58" containerID="07ce00f89f35e5a44ac66371fafb1d2ba3df21eebd1ac3e5357f4f2e0421e787" exitCode=0
Sep 11 01:13:41.275823 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:41.275665    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0b6ae428-7f54bc76c4-s27dn"
Sep 11 01:13:41.275823 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:41.275708    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0b6ae428-7f54bc76c4-s27dn" event={"ID":"d903fb9a-cd96-4c42-9dc8-6e4069e4eb58","Type":"ContainerDied","Data":"07ce00f89f35e5a44ac66371fafb1d2ba3df21eebd1ac3e5357f4f2e0421e787"}
Sep 11 01:13:41.275823 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:41.275760    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0b6ae428-7f54bc76c4-s27dn" event={"ID":"d903fb9a-cd96-4c42-9dc8-6e4069e4eb58","Type":"ContainerDied","Data":"fd9d38c645b9fcf004d5e6e011c750fc66566fbb2078508524e303eda1a90a1a"}
Sep 11 01:13:41.275823 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:41.275788    2574 scope.go:117] "RemoveContainer" containerID="07ce00f89f35e5a44ac66371fafb1d2ba3df21eebd1ac3e5357f4f2e0421e787"
Sep 11 01:13:41.277322 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:41.277302    2574 generic.go:356] "Generic (PLEG): container finished" podID="3e83ff0c-f75d-47ce-915e-cb475b4a0394" containerID="9042bd6d5eca969e07571cf8efe0f8d2efb7ccc9ce55971e2a5ce80d6f63424c" exitCode=0
Sep 11 01:13:41.277440 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:41.277358    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-0b6ae428-d9c99cfd-pgz6v"
Sep 11 01:13:41.277506 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:41.277360    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-0b6ae428-d9c99cfd-pgz6v" event={"ID":"3e83ff0c-f75d-47ce-915e-cb475b4a0394","Type":"ContainerDied","Data":"9042bd6d5eca969e07571cf8efe0f8d2efb7ccc9ce55971e2a5ce80d6f63424c"}
Sep 11 01:13:41.277506 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:41.277484    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-0b6ae428-d9c99cfd-pgz6v" event={"ID":"3e83ff0c-f75d-47ce-915e-cb475b4a0394","Type":"ContainerDied","Data":"84df1215cc449242dbd81db2359d6b2dd5c92df95b8acdad02789f342bba4fa0"}
Sep 11 01:13:41.286646 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:41.286626    2574 scope.go:117] "RemoveContainer" containerID="07ce00f89f35e5a44ac66371fafb1d2ba3df21eebd1ac3e5357f4f2e0421e787"
Sep 11 01:13:41.287099 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:13:41.287070    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07ce00f89f35e5a44ac66371fafb1d2ba3df21eebd1ac3e5357f4f2e0421e787\": container with ID starting with 07ce00f89f35e5a44ac66371fafb1d2ba3df21eebd1ac3e5357f4f2e0421e787 not found: ID does not exist" containerID="07ce00f89f35e5a44ac66371fafb1d2ba3df21eebd1ac3e5357f4f2e0421e787"
Sep 11 01:13:41.287198 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:41.287109    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07ce00f89f35e5a44ac66371fafb1d2ba3df21eebd1ac3e5357f4f2e0421e787"} err="failed to get container status \"07ce00f89f35e5a44ac66371fafb1d2ba3df21eebd1ac3e5357f4f2e0421e787\": rpc error: code = NotFound desc = could not find container \"07ce00f89f35e5a44ac66371fafb1d2ba3df21eebd1ac3e5357f4f2e0421e787\": container with ID starting with 07ce00f89f35e5a44ac66371fafb1d2ba3df21eebd1ac3e5357f4f2e0421e787 not found: ID does not exist"
Sep 11 01:13:41.287198 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:41.287125    2574 scope.go:117] "RemoveContainer" containerID="9042bd6d5eca969e07571cf8efe0f8d2efb7ccc9ce55971e2a5ce80d6f63424c"
Sep 11 01:13:41.295379 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:41.295349    2574 scope.go:117] "RemoveContainer" containerID="9042bd6d5eca969e07571cf8efe0f8d2efb7ccc9ce55971e2a5ce80d6f63424c"
Sep 11 01:13:41.295623 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:13:41.295602    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9042bd6d5eca969e07571cf8efe0f8d2efb7ccc9ce55971e2a5ce80d6f63424c\": container with ID starting with 9042bd6d5eca969e07571cf8efe0f8d2efb7ccc9ce55971e2a5ce80d6f63424c not found: ID does not exist" containerID="9042bd6d5eca969e07571cf8efe0f8d2efb7ccc9ce55971e2a5ce80d6f63424c"
Sep 11 01:13:41.295690 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:41.295633    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9042bd6d5eca969e07571cf8efe0f8d2efb7ccc9ce55971e2a5ce80d6f63424c"} err="failed to get container status \"9042bd6d5eca969e07571cf8efe0f8d2efb7ccc9ce55971e2a5ce80d6f63424c\": rpc error: code = NotFound desc = could not find container \"9042bd6d5eca969e07571cf8efe0f8d2efb7ccc9ce55971e2a5ce80d6f63424c\": container with ID starting with 9042bd6d5eca969e07571cf8efe0f8d2efb7ccc9ce55971e2a5ce80d6f63424c not found: ID does not exist"
Sep 11 01:13:41.297164 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:41.297147    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0b6ae428-d9c99cfd-pgz6v"]
Sep 11 01:13:41.303214 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:41.303194    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0b6ae428-d9c99cfd-pgz6v"]
Sep 11 01:13:41.313797 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:41.313771    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-0b6ae428-7f54bc76c4-s27dn"]
Sep 11 01:13:41.317627 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:41.317608    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-0b6ae428-7f54bc76c4-s27dn"]
Sep 11 01:13:41.368896 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:41.368836    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:41.691157 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:41.691127    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:41.995528 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:41.995446    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:43.100807 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:43.100776    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e83ff0c-f75d-47ce-915e-cb475b4a0394" path="/var/lib/kubelet/pods/3e83ff0c-f75d-47ce-915e-cb475b4a0394/volumes"
Sep 11 01:13:43.101210 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:43.101143    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d903fb9a-cd96-4c42-9dc8-6e4069e4eb58" path="/var/lib/kubelet/pods/d903fb9a-cd96-4c42-9dc8-6e4069e4eb58/volumes"
Sep 11 01:13:43.952821 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:43.952786    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:44.018605 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:44.018575    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:45.281406 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.281376    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:45.348261 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.348230    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:45.718732 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.718701    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:45.816060 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.816028    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:45.847555 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.847523    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9516b91d-754f94ffd6-m927q"]
Sep 11 01:13:45.847937 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.847896    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e83ff0c-f75d-47ce-915e-cb475b4a0394" containerName="payload-processing"
Sep 11 01:13:45.847937 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.847922    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e83ff0c-f75d-47ce-915e-cb475b4a0394" containerName="payload-processing"
Sep 11 01:13:45.848045 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.847945    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="927405cc-d4c0-48d9-82ac-8637fd369398" containerName="revoke-keys"
Sep 11 01:13:45.848045 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.847951    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="927405cc-d4c0-48d9-82ac-8637fd369398" containerName="revoke-keys"
Sep 11 01:13:45.848045 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.847963    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d903fb9a-cd96-4c42-9dc8-6e4069e4eb58" containerName="payload-pre-processing"
Sep 11 01:13:45.848045 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.847968    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d903fb9a-cd96-4c42-9dc8-6e4069e4eb58" containerName="payload-pre-processing"
Sep 11 01:13:45.848045 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.848041    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e83ff0c-f75d-47ce-915e-cb475b4a0394" containerName="payload-processing"
Sep 11 01:13:45.848197 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.848051    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="927405cc-d4c0-48d9-82ac-8637fd369398" containerName="revoke-keys"
Sep 11 01:13:45.848197 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.848057    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d903fb9a-cd96-4c42-9dc8-6e4069e4eb58" containerName="payload-pre-processing"
Sep 11 01:13:45.850946 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.850904    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-9516b91d-754f94ffd6-m927q"
Sep 11 01:13:45.853129 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.853105    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-9516b91d\""
Sep 11 01:13:45.862103 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.862083    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9516b91d-754f94ffd6-m927q"]
Sep 11 01:13:45.874180 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.874151    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q"]
Sep 11 01:13:45.876419 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.876400    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q"
Sep 11 01:13:45.878979 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.878963    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-cd176e\"/\"default-dockercfg-kwbgx\""
Sep 11 01:13:45.879072 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.879008    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-cd176e\"/\"kube-root-ca.crt\""
Sep 11 01:13:45.879123 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.879012    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-cd176e\"/\"openshift-service-ca.crt\""
Sep 11 01:13:45.879503 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.879488    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-cd176e\"/\"simulator-w4-cd176e-kserve-self-signed-certs\""
Sep 11 01:13:45.887183 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.887162    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9516b91d-7cfd4dc57-kvdfc"]
Sep 11 01:13:45.889282 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.889258    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9516b91d-7cfd4dc57-kvdfc"
Sep 11 01:13:45.890576 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.890554    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q"]
Sep 11 01:13:45.891586 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.891567    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-9516b91d-dockercfg-rncgj\""
Sep 11 01:13:45.899147 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.899126    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9516b91d-7cfd4dc57-kvdfc"]
Sep 11 01:13:45.914067 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.914037    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:45.963552 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.963522    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4cnq\" (UniqueName: \"kubernetes.io/projected/23de6245-8285-43ca-82a6-21bda5f507a7-kube-api-access-f4cnq\") pod \"payload-pre-processing-e2e-ait-9516b91d-754f94ffd6-m927q\" (UID: \"23de6245-8285-43ca-82a6-21bda5f507a7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9516b91d-754f94ffd6-m927q"
Sep 11 01:13:45.963709 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.963568    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15337767-f54e-4950-904d-01283098bf2b-model-cache\") pod \"simulator-w4-cd176e-kserve-79d6954b95-29d4q\" (UID: \"15337767-f54e-4950-904d-01283098bf2b\") " pod="e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q"
Sep 11 01:13:45.963709 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.963634    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72kmx\" (UniqueName: \"kubernetes.io/projected/ff917096-2e53-426f-8eaf-de586c17a3e8-kube-api-access-72kmx\") pod \"payload-processing-e2e-ait-9516b91d-7cfd4dc57-kvdfc\" (UID: \"ff917096-2e53-426f-8eaf-de586c17a3e8\") " pod="openshift-ingress/payload-processing-e2e-ait-9516b91d-7cfd4dc57-kvdfc"
Sep 11 01:13:45.963709 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.963677    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15337767-f54e-4950-904d-01283098bf2b-home\") pod \"simulator-w4-cd176e-kserve-79d6954b95-29d4q\" (UID: \"15337767-f54e-4950-904d-01283098bf2b\") " pod="e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q"
Sep 11 01:13:45.963709 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.963696    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gt45d\" (UniqueName: \"kubernetes.io/projected/15337767-f54e-4950-904d-01283098bf2b-kube-api-access-gt45d\") pod \"simulator-w4-cd176e-kserve-79d6954b95-29d4q\" (UID: \"15337767-f54e-4950-904d-01283098bf2b\") " pod="e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q"
Sep 11 01:13:45.963849 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.963713    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15337767-f54e-4950-904d-01283098bf2b-tmp-dir\") pod \"simulator-w4-cd176e-kserve-79d6954b95-29d4q\" (UID: \"15337767-f54e-4950-904d-01283098bf2b\") " pod="e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q"
Sep 11 01:13:45.963849 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.963731    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15337767-f54e-4950-904d-01283098bf2b-tls-certs\") pod \"simulator-w4-cd176e-kserve-79d6954b95-29d4q\" (UID: \"15337767-f54e-4950-904d-01283098bf2b\") " pod="e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q"
Sep 11 01:13:45.963849 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.963786    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23de6245-8285-43ca-82a6-21bda5f507a7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-9516b91d-754f94ffd6-m927q\" (UID: \"23de6245-8285-43ca-82a6-21bda5f507a7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9516b91d-754f94ffd6-m927q"
Sep 11 01:13:45.963849 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.963832    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff917096-2e53-426f-8eaf-de586c17a3e8-plugins-config-volume\") pod \"payload-processing-e2e-ait-9516b91d-7cfd4dc57-kvdfc\" (UID: \"ff917096-2e53-426f-8eaf-de586c17a3e8\") " pod="openshift-ingress/payload-processing-e2e-ait-9516b91d-7cfd4dc57-kvdfc"
Sep 11 01:13:45.963998 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:45.963863    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15337767-f54e-4950-904d-01283098bf2b-dshm\") pod \"simulator-w4-cd176e-kserve-79d6954b95-29d4q\" (UID: \"15337767-f54e-4950-904d-01283098bf2b\") " pod="e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q"
Sep 11 01:13:46.064735 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:46.064632    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4cnq\" (UniqueName: \"kubernetes.io/projected/23de6245-8285-43ca-82a6-21bda5f507a7-kube-api-access-f4cnq\") pod \"payload-pre-processing-e2e-ait-9516b91d-754f94ffd6-m927q\" (UID: \"23de6245-8285-43ca-82a6-21bda5f507a7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9516b91d-754f94ffd6-m927q"
Sep 11 01:13:46.064735 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:46.064692    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15337767-f54e-4950-904d-01283098bf2b-model-cache\") pod \"simulator-w4-cd176e-kserve-79d6954b95-29d4q\" (UID: \"15337767-f54e-4950-904d-01283098bf2b\") " pod="e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q"
Sep 11 01:13:46.065050 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:46.064861    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72kmx\" (UniqueName: \"kubernetes.io/projected/ff917096-2e53-426f-8eaf-de586c17a3e8-kube-api-access-72kmx\") pod \"payload-processing-e2e-ait-9516b91d-7cfd4dc57-kvdfc\" (UID: \"ff917096-2e53-426f-8eaf-de586c17a3e8\") " pod="openshift-ingress/payload-processing-e2e-ait-9516b91d-7cfd4dc57-kvdfc"
Sep 11 01:13:46.065050 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:46.064952    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15337767-f54e-4950-904d-01283098bf2b-home\") pod \"simulator-w4-cd176e-kserve-79d6954b95-29d4q\" (UID: \"15337767-f54e-4950-904d-01283098bf2b\") " pod="e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q"
Sep 11 01:13:46.065050 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:46.064973    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gt45d\" (UniqueName: \"kubernetes.io/projected/15337767-f54e-4950-904d-01283098bf2b-kube-api-access-gt45d\") pod \"simulator-w4-cd176e-kserve-79d6954b95-29d4q\" (UID: \"15337767-f54e-4950-904d-01283098bf2b\") " pod="e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q"
Sep 11 01:13:46.065050 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:46.064990    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15337767-f54e-4950-904d-01283098bf2b-tmp-dir\") pod \"simulator-w4-cd176e-kserve-79d6954b95-29d4q\" (UID: \"15337767-f54e-4950-904d-01283098bf2b\") " pod="e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q"
Sep 11 01:13:46.065050 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:46.065016    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15337767-f54e-4950-904d-01283098bf2b-tls-certs\") pod \"simulator-w4-cd176e-kserve-79d6954b95-29d4q\" (UID: \"15337767-f54e-4950-904d-01283098bf2b\") " pod="e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q"
Sep 11 01:13:46.065050 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:46.065035    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15337767-f54e-4950-904d-01283098bf2b-model-cache\") pod \"simulator-w4-cd176e-kserve-79d6954b95-29d4q\" (UID: \"15337767-f54e-4950-904d-01283098bf2b\") " pod="e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q"
Sep 11 01:13:46.065050 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:46.065046    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23de6245-8285-43ca-82a6-21bda5f507a7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-9516b91d-754f94ffd6-m927q\" (UID: \"23de6245-8285-43ca-82a6-21bda5f507a7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9516b91d-754f94ffd6-m927q"
Sep 11 01:13:46.065462 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:46.065115    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff917096-2e53-426f-8eaf-de586c17a3e8-plugins-config-volume\") pod \"payload-processing-e2e-ait-9516b91d-7cfd4dc57-kvdfc\" (UID: \"ff917096-2e53-426f-8eaf-de586c17a3e8\") " pod="openshift-ingress/payload-processing-e2e-ait-9516b91d-7cfd4dc57-kvdfc"
Sep 11 01:13:46.065462 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:46.065164    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15337767-f54e-4950-904d-01283098bf2b-dshm\") pod \"simulator-w4-cd176e-kserve-79d6954b95-29d4q\" (UID: \"15337767-f54e-4950-904d-01283098bf2b\") " pod="e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q"
Sep 11 01:13:46.065462 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:46.065291    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15337767-f54e-4950-904d-01283098bf2b-home\") pod \"simulator-w4-cd176e-kserve-79d6954b95-29d4q\" (UID: \"15337767-f54e-4950-904d-01283098bf2b\") " pod="e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q"
Sep 11 01:13:46.065462 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:46.065349    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15337767-f54e-4950-904d-01283098bf2b-tmp-dir\") pod \"simulator-w4-cd176e-kserve-79d6954b95-29d4q\" (UID: \"15337767-f54e-4950-904d-01283098bf2b\") " pod="e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q"
Sep 11 01:13:46.066057 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:46.066012    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff917096-2e53-426f-8eaf-de586c17a3e8-plugins-config-volume\") pod \"payload-processing-e2e-ait-9516b91d-7cfd4dc57-kvdfc\" (UID: \"ff917096-2e53-426f-8eaf-de586c17a3e8\") " pod="openshift-ingress/payload-processing-e2e-ait-9516b91d-7cfd4dc57-kvdfc"
Sep 11 01:13:46.066057 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:46.066022    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23de6245-8285-43ca-82a6-21bda5f507a7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-9516b91d-754f94ffd6-m927q\" (UID: \"23de6245-8285-43ca-82a6-21bda5f507a7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9516b91d-754f94ffd6-m927q"
Sep 11 01:13:46.068237 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:46.068214    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15337767-f54e-4950-904d-01283098bf2b-tls-certs\") pod \"simulator-w4-cd176e-kserve-79d6954b95-29d4q\" (UID: \"15337767-f54e-4950-904d-01283098bf2b\") " pod="e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q"
Sep 11 01:13:46.068651 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:46.068631    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15337767-f54e-4950-904d-01283098bf2b-dshm\") pod \"simulator-w4-cd176e-kserve-79d6954b95-29d4q\" (UID: \"15337767-f54e-4950-904d-01283098bf2b\") " pod="e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q"
Sep 11 01:13:46.073511 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:46.073487    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gt45d\" (UniqueName: \"kubernetes.io/projected/15337767-f54e-4950-904d-01283098bf2b-kube-api-access-gt45d\") pod \"simulator-w4-cd176e-kserve-79d6954b95-29d4q\" (UID: \"15337767-f54e-4950-904d-01283098bf2b\") " pod="e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q"
Sep 11 01:13:46.074073 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:46.074049    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4cnq\" (UniqueName: \"kubernetes.io/projected/23de6245-8285-43ca-82a6-21bda5f507a7-kube-api-access-f4cnq\") pod \"payload-pre-processing-e2e-ait-9516b91d-754f94ffd6-m927q\" (UID: \"23de6245-8285-43ca-82a6-21bda5f507a7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9516b91d-754f94ffd6-m927q"
Sep 11 01:13:46.074254 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:46.074235    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72kmx\" (UniqueName: \"kubernetes.io/projected/ff917096-2e53-426f-8eaf-de586c17a3e8-kube-api-access-72kmx\") pod \"payload-processing-e2e-ait-9516b91d-7cfd4dc57-kvdfc\" (UID: \"ff917096-2e53-426f-8eaf-de586c17a3e8\") " pod="openshift-ingress/payload-processing-e2e-ait-9516b91d-7cfd4dc57-kvdfc"
Sep 11 01:13:46.098863 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:46.098840    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:46.161771 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:46.161746    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-9516b91d-754f94ffd6-m927q"
Sep 11 01:13:46.186308 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:46.186278    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q"
Sep 11 01:13:46.201287 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:46.201255    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9516b91d-7cfd4dc57-kvdfc"
Sep 11 01:13:46.372815 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:46.372744    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:46.442567 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:46.442545    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9516b91d-754f94ffd6-m927q"]
Sep 11 01:13:46.444507 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:13:46.444477    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23de6245_8285_43ca_82a6_21bda5f507a7.slice/crio-94c2c44a2721fd1d02b88ec3448343e8ad603b665da87a942fcb14b1890fc498 WatchSource:0}: Error finding container 94c2c44a2721fd1d02b88ec3448343e8ad603b665da87a942fcb14b1890fc498: Status 404 returned error can't find the container with id 94c2c44a2721fd1d02b88ec3448343e8ad603b665da87a942fcb14b1890fc498
Sep 11 01:13:46.479188 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:46.479152    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q"]
Sep 11 01:13:46.480272 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:13:46.480249    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod15337767_f54e_4950_904d_01283098bf2b.slice/crio-ec1bad9a0208fcdde59d46a322c084501a99c13c113b2528cb717607561b7088 WatchSource:0}: Error finding container ec1bad9a0208fcdde59d46a322c084501a99c13c113b2528cb717607561b7088: Status 404 returned error can't find the container with id ec1bad9a0208fcdde59d46a322c084501a99c13c113b2528cb717607561b7088
Sep 11 01:13:46.495219 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:46.495182    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9516b91d-7cfd4dc57-kvdfc"]
Sep 11 01:13:46.497079 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:13:46.497056    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff917096_2e53_426f_8eaf_de586c17a3e8.slice/crio-2643fb20b4a1e9523903e516b035b3a47a02003029246c9fb951b6b05dc4900d WatchSource:0}: Error finding container 2643fb20b4a1e9523903e516b035b3a47a02003029246c9fb951b6b05dc4900d: Status 404 returned error can't find the container with id 2643fb20b4a1e9523903e516b035b3a47a02003029246c9fb951b6b05dc4900d
Sep 11 01:13:47.311974 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:47.311930    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9516b91d-7cfd4dc57-kvdfc" event={"ID":"ff917096-2e53-426f-8eaf-de586c17a3e8","Type":"ContainerStarted","Data":"eeef844418596d2c4a27d6b623ca2808204f1e40b399530d8d45120106474f1e"}
Sep 11 01:13:47.312187 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:47.311998    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9516b91d-7cfd4dc57-kvdfc" event={"ID":"ff917096-2e53-426f-8eaf-de586c17a3e8","Type":"ContainerStarted","Data":"2643fb20b4a1e9523903e516b035b3a47a02003029246c9fb951b6b05dc4900d"}
Sep 11 01:13:47.312187 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:47.312024    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-9516b91d-7cfd4dc57-kvdfc"
Sep 11 01:13:47.313553 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:47.313526    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q" event={"ID":"15337767-f54e-4950-904d-01283098bf2b","Type":"ContainerStarted","Data":"ca4fa4a178459b9c2deea936b6a3d86cdf6a11f897c4385f4fe893408fd17ce6"}
Sep 11 01:13:47.313677 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:47.313561    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q" event={"ID":"15337767-f54e-4950-904d-01283098bf2b","Type":"ContainerStarted","Data":"ec1bad9a0208fcdde59d46a322c084501a99c13c113b2528cb717607561b7088"}
Sep 11 01:13:47.315461 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:47.315430    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-9516b91d-754f94ffd6-m927q" event={"ID":"23de6245-8285-43ca-82a6-21bda5f507a7","Type":"ContainerStarted","Data":"80a3f046e507b4ce30fd5688f57b8b0a7152323631d6a224f39ddae6711e3b55"}
Sep 11 01:13:47.315548 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:47.315486    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-9516b91d-754f94ffd6-m927q" event={"ID":"23de6245-8285-43ca-82a6-21bda5f507a7","Type":"ContainerStarted","Data":"94c2c44a2721fd1d02b88ec3448343e8ad603b665da87a942fcb14b1890fc498"}
Sep 11 01:13:47.315628 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:47.315609    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-9516b91d-754f94ffd6-m927q"
Sep 11 01:13:47.331640 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:47.331600    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-9516b91d-7cfd4dc57-kvdfc" podStartSLOduration=2.331585696 podStartE2EDuration="2.331585696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:13:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:13:47.32951765 +0000 UTC m=+942.726709824" watchObservedRunningTime="2026-09-11 01:13:47.331585696 +0000 UTC m=+942.728777869"
Sep 11 01:13:47.344708 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:47.344654    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-9516b91d-754f94ffd6-m927q" podStartSLOduration=2.344634794 podStartE2EDuration="2.344634794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:13:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:13:47.344589386 +0000 UTC m=+942.741781559" watchObservedRunningTime="2026-09-11 01:13:47.344634794 +0000 UTC m=+942.741826969"
Sep 11 01:13:47.361503 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:47.361443    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q" podStartSLOduration=2.361420559 podStartE2EDuration="2.361420559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:13:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:13:47.359624678 +0000 UTC m=+942.756816852" watchObservedRunningTime="2026-09-11 01:13:47.361420559 +0000 UTC m=+942.758612735"
Sep 11 01:13:47.535536 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:47.535498    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:13:56.187005 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:56.186969    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q"
Sep 11 01:13:56.187380 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:56.187020    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q"
Sep 11 01:13:56.199611 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:56.199587    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q"
Sep 11 01:13:56.372509 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:56.372472    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q"
Sep 11 01:13:58.322309 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:58.322272    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-9516b91d-7cfd4dc57-kvdfc"
Sep 11 01:13:58.322709 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:13:58.322440    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-9516b91d-754f94ffd6-m927q"
Sep 11 01:14:06.148127 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:06.148092    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:09.101091 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:09.101064    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:09.872499 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:09.872461    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9516b91d-lppzp"]
Sep 11 01:14:09.877686 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:09.877654    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9516b91d-lppzp"
Sep 11 01:14:09.880139 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:09.880107    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 01:14:09.880278 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:09.880172    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q6gbn\""
Sep 11 01:14:09.882531 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:09.882498    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9516b91d-lppzp"]
Sep 11 01:14:09.980845 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:09.980820    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18d9bbf5-69b9-46a4-97e0-c905abeeb0fb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9516b91d-lppzp\" (UID: \"18d9bbf5-69b9-46a4-97e0-c905abeeb0fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9516b91d-lppzp"
Sep 11 01:14:10.081996 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:10.081969    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18d9bbf5-69b9-46a4-97e0-c905abeeb0fb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9516b91d-lppzp\" (UID: \"18d9bbf5-69b9-46a4-97e0-c905abeeb0fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9516b91d-lppzp"
Sep 11 01:14:10.082558 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:10.082537    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18d9bbf5-69b9-46a4-97e0-c905abeeb0fb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9516b91d-lppzp\" (UID: \"18d9bbf5-69b9-46a4-97e0-c905abeeb0fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9516b91d-lppzp"
Sep 11 01:14:10.190545 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:10.190523    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9516b91d-lppzp"
Sep 11 01:14:10.460596 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:10.460430    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9516b91d-lppzp"]
Sep 11 01:14:10.462885 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:14:10.462854    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18d9bbf5_69b9_46a4_97e0_c905abeeb0fb.slice/crio-acbb1f7bd33bb2e528fac415ea1f99f97fed300d8cb643ecb537188491cb5687 WatchSource:0}: Error finding container acbb1f7bd33bb2e528fac415ea1f99f97fed300d8cb643ecb537188491cb5687: Status 404 returned error can't find the container with id acbb1f7bd33bb2e528fac415ea1f99f97fed300d8cb643ecb537188491cb5687
Sep 11 01:14:11.439317 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:11.439285    2574 generic.go:356] "Generic (PLEG): container finished" podID="18d9bbf5-69b9-46a4-97e0-c905abeeb0fb" containerID="45447558de0135b18547fd79fe712a7153f5144290d638d812ad28ca9d46403d" exitCode=0
Sep 11 01:14:11.439678 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:11.439377    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9516b91d-lppzp" event={"ID":"18d9bbf5-69b9-46a4-97e0-c905abeeb0fb","Type":"ContainerDied","Data":"45447558de0135b18547fd79fe712a7153f5144290d638d812ad28ca9d46403d"}
Sep 11 01:14:11.439678 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:11.439431    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9516b91d-lppzp" event={"ID":"18d9bbf5-69b9-46a4-97e0-c905abeeb0fb","Type":"ContainerStarted","Data":"acbb1f7bd33bb2e528fac415ea1f99f97fed300d8cb643ecb537188491cb5687"}
Sep 11 01:14:12.573024 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:12.573001    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9516b91d-lppzp"
Sep 11 01:14:12.706441 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:12.706359    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18d9bbf5-69b9-46a4-97e0-c905abeeb0fb-maas-api-service-ca\") pod \"18d9bbf5-69b9-46a4-97e0-c905abeeb0fb\" (UID: \"18d9bbf5-69b9-46a4-97e0-c905abeeb0fb\") "
Sep 11 01:14:12.706759 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:12.706724    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18d9bbf5-69b9-46a4-97e0-c905abeeb0fb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "18d9bbf5-69b9-46a4-97e0-c905abeeb0fb" (UID: "18d9bbf5-69b9-46a4-97e0-c905abeeb0fb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:14:12.807954 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:12.807901    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18d9bbf5-69b9-46a4-97e0-c905abeeb0fb-maas-api-service-ca\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:14:13.450753 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:13.450728    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9516b91d-lppzp"
Sep 11 01:14:13.450960 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:13.450774    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9516b91d-lppzp" event={"ID":"18d9bbf5-69b9-46a4-97e0-c905abeeb0fb","Type":"ContainerDied","Data":"acbb1f7bd33bb2e528fac415ea1f99f97fed300d8cb643ecb537188491cb5687"}
Sep 11 01:14:13.450960 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:13.450835    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="acbb1f7bd33bb2e528fac415ea1f99f97fed300d8cb643ecb537188491cb5687"
Sep 11 01:14:15.786062 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:15.786004    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:15.811107 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:15.811074    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9516b91d-7cfd4dc57-kvdfc"]
Sep 11 01:14:15.811286 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:15.811264    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-9516b91d-7cfd4dc57-kvdfc" podUID="ff917096-2e53-426f-8eaf-de586c17a3e8" containerName="payload-processing" containerID="cri-o://eeef844418596d2c4a27d6b623ca2808204f1e40b399530d8d45120106474f1e" gracePeriod=30
Sep 11 01:14:15.842042 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:15.841996    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9516b91d-754f94ffd6-m927q"]
Sep 11 01:14:15.842506 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:15.842446    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-9516b91d-754f94ffd6-m927q" podUID="23de6245-8285-43ca-82a6-21bda5f507a7" containerName="payload-pre-processing" containerID="cri-o://80a3f046e507b4ce30fd5688f57b8b0a7152323631d6a224f39ddae6711e3b55" gracePeriod=30
Sep 11 01:14:15.887936 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:15.866139    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:15.931273 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:15.931235    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:15.991060 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:15.990983    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9516b91d-lppzp"]
Sep 11 01:14:15.995615 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:15.995584    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9516b91d-lppzp"]
Sep 11 01:14:16.002052 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:16.002015    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:16.061387 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:16.061299    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:16.112453 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:16.112421    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9516b91d-7cfd4dc57-kvdfc"
Sep 11 01:14:16.142251 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:16.142222    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-9516b91d-754f94ffd6-m927q"
Sep 11 01:14:16.239654 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:16.239624    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72kmx\" (UniqueName: \"kubernetes.io/projected/ff917096-2e53-426f-8eaf-de586c17a3e8-kube-api-access-72kmx\") pod \"ff917096-2e53-426f-8eaf-de586c17a3e8\" (UID: \"ff917096-2e53-426f-8eaf-de586c17a3e8\") "
Sep 11 01:14:16.239854 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:16.239690    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff917096-2e53-426f-8eaf-de586c17a3e8-plugins-config-volume\") pod \"ff917096-2e53-426f-8eaf-de586c17a3e8\" (UID: \"ff917096-2e53-426f-8eaf-de586c17a3e8\") "
Sep 11 01:14:16.240197 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:16.240168    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff917096-2e53-426f-8eaf-de586c17a3e8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ff917096-2e53-426f-8eaf-de586c17a3e8" (UID: "ff917096-2e53-426f-8eaf-de586c17a3e8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:14:16.241975 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:16.241952    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff917096-2e53-426f-8eaf-de586c17a3e8-kube-api-access-72kmx" (OuterVolumeSpecName: "kube-api-access-72kmx") pod "ff917096-2e53-426f-8eaf-de586c17a3e8" (UID: "ff917096-2e53-426f-8eaf-de586c17a3e8"). InnerVolumeSpecName "kube-api-access-72kmx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:14:16.341274 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:16.341176    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23de6245-8285-43ca-82a6-21bda5f507a7-plugins-config-volume\") pod \"23de6245-8285-43ca-82a6-21bda5f507a7\" (UID: \"23de6245-8285-43ca-82a6-21bda5f507a7\") "
Sep 11 01:14:16.341274 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:16.341223    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4cnq\" (UniqueName: \"kubernetes.io/projected/23de6245-8285-43ca-82a6-21bda5f507a7-kube-api-access-f4cnq\") pod \"23de6245-8285-43ca-82a6-21bda5f507a7\" (UID: \"23de6245-8285-43ca-82a6-21bda5f507a7\") "
Sep 11 01:14:16.341515 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:16.341394    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff917096-2e53-426f-8eaf-de586c17a3e8-plugins-config-volume\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:14:16.341515 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:16.341408    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72kmx\" (UniqueName: \"kubernetes.io/projected/ff917096-2e53-426f-8eaf-de586c17a3e8-kube-api-access-72kmx\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:14:16.341623 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:16.341538    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23de6245-8285-43ca-82a6-21bda5f507a7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "23de6245-8285-43ca-82a6-21bda5f507a7" (UID: "23de6245-8285-43ca-82a6-21bda5f507a7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:14:16.343670 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:16.343637    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23de6245-8285-43ca-82a6-21bda5f507a7-kube-api-access-f4cnq" (OuterVolumeSpecName: "kube-api-access-f4cnq") pod "23de6245-8285-43ca-82a6-21bda5f507a7" (UID: "23de6245-8285-43ca-82a6-21bda5f507a7"). InnerVolumeSpecName "kube-api-access-f4cnq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:14:16.442713 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:16.442674    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23de6245-8285-43ca-82a6-21bda5f507a7-plugins-config-volume\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:14:16.442713 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:16.442704    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4cnq\" (UniqueName: \"kubernetes.io/projected/23de6245-8285-43ca-82a6-21bda5f507a7-kube-api-access-f4cnq\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:14:16.467605 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:16.467573    2574 generic.go:356] "Generic (PLEG): container finished" podID="23de6245-8285-43ca-82a6-21bda5f507a7" containerID="80a3f046e507b4ce30fd5688f57b8b0a7152323631d6a224f39ddae6711e3b55" exitCode=0
Sep 11 01:14:16.467793 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:16.467629    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-9516b91d-754f94ffd6-m927q"
Sep 11 01:14:16.467793 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:16.467675    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-9516b91d-754f94ffd6-m927q" event={"ID":"23de6245-8285-43ca-82a6-21bda5f507a7","Type":"ContainerDied","Data":"80a3f046e507b4ce30fd5688f57b8b0a7152323631d6a224f39ddae6711e3b55"}
Sep 11 01:14:16.467793 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:16.467741    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-9516b91d-754f94ffd6-m927q" event={"ID":"23de6245-8285-43ca-82a6-21bda5f507a7","Type":"ContainerDied","Data":"94c2c44a2721fd1d02b88ec3448343e8ad603b665da87a942fcb14b1890fc498"}
Sep 11 01:14:16.467793 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:16.467775    2574 scope.go:117] "RemoveContainer" containerID="80a3f046e507b4ce30fd5688f57b8b0a7152323631d6a224f39ddae6711e3b55"
Sep 11 01:14:16.469273 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:16.469250    2574 generic.go:356] "Generic (PLEG): container finished" podID="ff917096-2e53-426f-8eaf-de586c17a3e8" containerID="eeef844418596d2c4a27d6b623ca2808204f1e40b399530d8d45120106474f1e" exitCode=0
Sep 11 01:14:16.469407 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:16.469277    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9516b91d-7cfd4dc57-kvdfc" event={"ID":"ff917096-2e53-426f-8eaf-de586c17a3e8","Type":"ContainerDied","Data":"eeef844418596d2c4a27d6b623ca2808204f1e40b399530d8d45120106474f1e"}
Sep 11 01:14:16.469407 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:16.469301    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9516b91d-7cfd4dc57-kvdfc"
Sep 11 01:14:16.469407 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:16.469317    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9516b91d-7cfd4dc57-kvdfc" event={"ID":"ff917096-2e53-426f-8eaf-de586c17a3e8","Type":"ContainerDied","Data":"2643fb20b4a1e9523903e516b035b3a47a02003029246c9fb951b6b05dc4900d"}
Sep 11 01:14:16.483086 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:16.480148    2574 scope.go:117] "RemoveContainer" containerID="80a3f046e507b4ce30fd5688f57b8b0a7152323631d6a224f39ddae6711e3b55"
Sep 11 01:14:16.483086 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:14:16.480790    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80a3f046e507b4ce30fd5688f57b8b0a7152323631d6a224f39ddae6711e3b55\": container with ID starting with 80a3f046e507b4ce30fd5688f57b8b0a7152323631d6a224f39ddae6711e3b55 not found: ID does not exist" containerID="80a3f046e507b4ce30fd5688f57b8b0a7152323631d6a224f39ddae6711e3b55"
Sep 11 01:14:16.483086 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:16.480841    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80a3f046e507b4ce30fd5688f57b8b0a7152323631d6a224f39ddae6711e3b55"} err="failed to get container status \"80a3f046e507b4ce30fd5688f57b8b0a7152323631d6a224f39ddae6711e3b55\": rpc error: code = NotFound desc = could not find container \"80a3f046e507b4ce30fd5688f57b8b0a7152323631d6a224f39ddae6711e3b55\": container with ID starting with 80a3f046e507b4ce30fd5688f57b8b0a7152323631d6a224f39ddae6711e3b55 not found: ID does not exist"
Sep 11 01:14:16.483086 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:16.480865    2574 scope.go:117] "RemoveContainer" containerID="eeef844418596d2c4a27d6b623ca2808204f1e40b399530d8d45120106474f1e"
Sep 11 01:14:16.490210 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:16.490193    2574 scope.go:117] "RemoveContainer" containerID="eeef844418596d2c4a27d6b623ca2808204f1e40b399530d8d45120106474f1e"
Sep 11 01:14:16.490495 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:14:16.490473    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eeef844418596d2c4a27d6b623ca2808204f1e40b399530d8d45120106474f1e\": container with ID starting with eeef844418596d2c4a27d6b623ca2808204f1e40b399530d8d45120106474f1e not found: ID does not exist" containerID="eeef844418596d2c4a27d6b623ca2808204f1e40b399530d8d45120106474f1e"
Sep 11 01:14:16.490587 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:16.490499    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eeef844418596d2c4a27d6b623ca2808204f1e40b399530d8d45120106474f1e"} err="failed to get container status \"eeef844418596d2c4a27d6b623ca2808204f1e40b399530d8d45120106474f1e\": rpc error: code = NotFound desc = could not find container \"eeef844418596d2c4a27d6b623ca2808204f1e40b399530d8d45120106474f1e\": container with ID starting with eeef844418596d2c4a27d6b623ca2808204f1e40b399530d8d45120106474f1e not found: ID does not exist"
Sep 11 01:14:16.500806 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:16.500777    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9516b91d-754f94ffd6-m927q"]
Sep 11 01:14:16.507438 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:16.507415    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9516b91d-754f94ffd6-m927q"]
Sep 11 01:14:16.523216 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:16.523184    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9516b91d-7cfd4dc57-kvdfc"]
Sep 11 01:14:16.533358 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:16.533334    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9516b91d-7cfd4dc57-kvdfc"]
Sep 11 01:14:17.101488 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:17.101406    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18d9bbf5-69b9-46a4-97e0-c905abeeb0fb" path="/var/lib/kubelet/pods/18d9bbf5-69b9-46a4-97e0-c905abeeb0fb/volumes"
Sep 11 01:14:17.101844 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:17.101757    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23de6245-8285-43ca-82a6-21bda5f507a7" path="/var/lib/kubelet/pods/23de6245-8285-43ca-82a6-21bda5f507a7/volumes"
Sep 11 01:14:17.102117 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:17.102102    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff917096-2e53-426f-8eaf-de586c17a3e8" path="/var/lib/kubelet/pods/ff917096-2e53-426f-8eaf-de586c17a3e8/volumes"
Sep 11 01:14:17.209783 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:17.209726    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:17.672185 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:17.672145    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:17.988644 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:17.988564    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:18.881474 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:18.881446    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:20.183405 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:20.183365    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:20.277228 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:20.277197    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:21.330449 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:21.330391    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:21.434920 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:21.434877    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:21.795515 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:21.795484    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:21.878848 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:21.878816    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:21.885978 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:21.885952    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6573d049-69dfd459bb-cmfnr"]
Sep 11 01:14:21.886373 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:21.886360    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18d9bbf5-69b9-46a4-97e0-c905abeeb0fb" containerName="revoke-keys"
Sep 11 01:14:21.886440 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:21.886374    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="18d9bbf5-69b9-46a4-97e0-c905abeeb0fb" containerName="revoke-keys"
Sep 11 01:14:21.886440 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:21.886398    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23de6245-8285-43ca-82a6-21bda5f507a7" containerName="payload-pre-processing"
Sep 11 01:14:21.886440 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:21.886414    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="23de6245-8285-43ca-82a6-21bda5f507a7" containerName="payload-pre-processing"
Sep 11 01:14:21.886440 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:21.886425    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff917096-2e53-426f-8eaf-de586c17a3e8" containerName="payload-processing"
Sep 11 01:14:21.886440 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:21.886431    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff917096-2e53-426f-8eaf-de586c17a3e8" containerName="payload-processing"
Sep 11 01:14:21.886696 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:21.886496    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff917096-2e53-426f-8eaf-de586c17a3e8" containerName="payload-processing"
Sep 11 01:14:21.886696 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:21.886507    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="23de6245-8285-43ca-82a6-21bda5f507a7" containerName="payload-pre-processing"
Sep 11 01:14:21.886696 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:21.886516    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="18d9bbf5-69b9-46a4-97e0-c905abeeb0fb" containerName="revoke-keys"
Sep 11 01:14:21.889486 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:21.889471    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-6573d049-69dfd459bb-cmfnr"
Sep 11 01:14:21.891678 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:21.891659    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-6573d049\""
Sep 11 01:14:21.899184 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:21.899162    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6573d049-69dfd459bb-cmfnr"]
Sep 11 01:14:21.920741 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:21.920711    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-6573d049-788d8d7cdf-v2zxr"]
Sep 11 01:14:21.923030 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:21.923005    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-6573d049-788d8d7cdf-v2zxr"
Sep 11 01:14:21.925361 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:21.925333    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-6573d049-dockercfg-7mzhh\""
Sep 11 01:14:21.938309 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:21.938288    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-6573d049-788d8d7cdf-v2zxr"]
Sep 11 01:14:21.984824 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:21.984796    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a38c1497-facc-4a2b-93d3-fa7d1dc28f50-plugins-config-volume\") pod \"payload-processing-e2e-derive-6573d049-788d8d7cdf-v2zxr\" (UID: \"a38c1497-facc-4a2b-93d3-fa7d1dc28f50\") " pod="openshift-ingress/payload-processing-e2e-derive-6573d049-788d8d7cdf-v2zxr"
Sep 11 01:14:21.984981 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:21.984861    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lrgp\" (UniqueName: \"kubernetes.io/projected/a38c1497-facc-4a2b-93d3-fa7d1dc28f50-kube-api-access-9lrgp\") pod \"payload-processing-e2e-derive-6573d049-788d8d7cdf-v2zxr\" (UID: \"a38c1497-facc-4a2b-93d3-fa7d1dc28f50\") " pod="openshift-ingress/payload-processing-e2e-derive-6573d049-788d8d7cdf-v2zxr"
Sep 11 01:14:21.984981 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:21.984903    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jn7nz\" (UniqueName: \"kubernetes.io/projected/db358587-cd47-4b7c-8667-8641b2af5334-kube-api-access-jn7nz\") pod \"payload-pre-processing-e2e-derive-6573d049-69dfd459bb-cmfnr\" (UID: \"db358587-cd47-4b7c-8667-8641b2af5334\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6573d049-69dfd459bb-cmfnr"
Sep 11 01:14:21.985077 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:21.985009    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db358587-cd47-4b7c-8667-8641b2af5334-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-6573d049-69dfd459bb-cmfnr\" (UID: \"db358587-cd47-4b7c-8667-8641b2af5334\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6573d049-69dfd459bb-cmfnr"
Sep 11 01:14:22.085777 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:22.085698    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lrgp\" (UniqueName: \"kubernetes.io/projected/a38c1497-facc-4a2b-93d3-fa7d1dc28f50-kube-api-access-9lrgp\") pod \"payload-processing-e2e-derive-6573d049-788d8d7cdf-v2zxr\" (UID: \"a38c1497-facc-4a2b-93d3-fa7d1dc28f50\") " pod="openshift-ingress/payload-processing-e2e-derive-6573d049-788d8d7cdf-v2zxr"
Sep 11 01:14:22.085777 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:22.085740    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jn7nz\" (UniqueName: \"kubernetes.io/projected/db358587-cd47-4b7c-8667-8641b2af5334-kube-api-access-jn7nz\") pod \"payload-pre-processing-e2e-derive-6573d049-69dfd459bb-cmfnr\" (UID: \"db358587-cd47-4b7c-8667-8641b2af5334\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6573d049-69dfd459bb-cmfnr"
Sep 11 01:14:22.086005 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:22.085789    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db358587-cd47-4b7c-8667-8641b2af5334-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-6573d049-69dfd459bb-cmfnr\" (UID: \"db358587-cd47-4b7c-8667-8641b2af5334\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6573d049-69dfd459bb-cmfnr"
Sep 11 01:14:22.086005 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:22.085820    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a38c1497-facc-4a2b-93d3-fa7d1dc28f50-plugins-config-volume\") pod \"payload-processing-e2e-derive-6573d049-788d8d7cdf-v2zxr\" (UID: \"a38c1497-facc-4a2b-93d3-fa7d1dc28f50\") " pod="openshift-ingress/payload-processing-e2e-derive-6573d049-788d8d7cdf-v2zxr"
Sep 11 01:14:22.086573 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:22.086552    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db358587-cd47-4b7c-8667-8641b2af5334-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-6573d049-69dfd459bb-cmfnr\" (UID: \"db358587-cd47-4b7c-8667-8641b2af5334\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6573d049-69dfd459bb-cmfnr"
Sep 11 01:14:22.086632 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:22.086558    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a38c1497-facc-4a2b-93d3-fa7d1dc28f50-plugins-config-volume\") pod \"payload-processing-e2e-derive-6573d049-788d8d7cdf-v2zxr\" (UID: \"a38c1497-facc-4a2b-93d3-fa7d1dc28f50\") " pod="openshift-ingress/payload-processing-e2e-derive-6573d049-788d8d7cdf-v2zxr"
Sep 11 01:14:22.094560 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:22.094539    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jn7nz\" (UniqueName: \"kubernetes.io/projected/db358587-cd47-4b7c-8667-8641b2af5334-kube-api-access-jn7nz\") pod \"payload-pre-processing-e2e-derive-6573d049-69dfd459bb-cmfnr\" (UID: \"db358587-cd47-4b7c-8667-8641b2af5334\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6573d049-69dfd459bb-cmfnr"
Sep 11 01:14:22.094690 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:22.094637    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lrgp\" (UniqueName: \"kubernetes.io/projected/a38c1497-facc-4a2b-93d3-fa7d1dc28f50-kube-api-access-9lrgp\") pod \"payload-processing-e2e-derive-6573d049-788d8d7cdf-v2zxr\" (UID: \"a38c1497-facc-4a2b-93d3-fa7d1dc28f50\") " pod="openshift-ingress/payload-processing-e2e-derive-6573d049-788d8d7cdf-v2zxr"
Sep 11 01:14:22.201476 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:22.201441    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-6573d049-69dfd459bb-cmfnr"
Sep 11 01:14:22.233004 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:22.232963    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-6573d049-788d8d7cdf-v2zxr"
Sep 11 01:14:22.378326 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:22.378227    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:22.478057 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:22.478022    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6573d049-69dfd459bb-cmfnr"]
Sep 11 01:14:22.478565 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:14:22.478543    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb358587_cd47_4b7c_8667_8641b2af5334.slice/crio-5f246e1df4d8542d7cb43c2449c1742c8f2f00d1e14b6b9edefb3136e9ea311f WatchSource:0}: Error finding container 5f246e1df4d8542d7cb43c2449c1742c8f2f00d1e14b6b9edefb3136e9ea311f: Status 404 returned error can't find the container with id 5f246e1df4d8542d7cb43c2449c1742c8f2f00d1e14b6b9edefb3136e9ea311f
Sep 11 01:14:22.502969 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:22.502946    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-6573d049-69dfd459bb-cmfnr" event={"ID":"db358587-cd47-4b7c-8667-8641b2af5334","Type":"ContainerStarted","Data":"5f246e1df4d8542d7cb43c2449c1742c8f2f00d1e14b6b9edefb3136e9ea311f"}
Sep 11 01:14:22.515637 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:22.515613    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-6573d049-788d8d7cdf-v2zxr"]
Sep 11 01:14:22.519150 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:14:22.519128    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda38c1497_facc_4a2b_93d3_fa7d1dc28f50.slice/crio-e0cf31e324ac66ea04d50ca5e2830b7a9a9f47e42d0f9d08816042e9df637e05 WatchSource:0}: Error finding container e0cf31e324ac66ea04d50ca5e2830b7a9a9f47e42d0f9d08816042e9df637e05: Status 404 returned error can't find the container with id e0cf31e324ac66ea04d50ca5e2830b7a9a9f47e42d0f9d08816042e9df637e05
Sep 11 01:14:23.509579 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:23.509539    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-6573d049-69dfd459bb-cmfnr" event={"ID":"db358587-cd47-4b7c-8667-8641b2af5334","Type":"ContainerStarted","Data":"30e6aa53e5e7066dee2e87e1181220314146a32f60904ed6b0deef5de97cfef0"}
Sep 11 01:14:23.510064 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:23.509689    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-6573d049-69dfd459bb-cmfnr"
Sep 11 01:14:23.511122 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:23.511101    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-6573d049-788d8d7cdf-v2zxr" event={"ID":"a38c1497-facc-4a2b-93d3-fa7d1dc28f50","Type":"ContainerStarted","Data":"7596ba725bc826cec5c0c1c06b6e9079ef4418679454e67eeaf691fc1fdef8c1"}
Sep 11 01:14:23.511234 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:23.511134    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-6573d049-788d8d7cdf-v2zxr" event={"ID":"a38c1497-facc-4a2b-93d3-fa7d1dc28f50","Type":"ContainerStarted","Data":"e0cf31e324ac66ea04d50ca5e2830b7a9a9f47e42d0f9d08816042e9df637e05"}
Sep 11 01:14:23.511234 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:23.511154    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-6573d049-788d8d7cdf-v2zxr"
Sep 11 01:14:23.525800 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:23.525752    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-6573d049-69dfd459bb-cmfnr" podStartSLOduration=2.52573826 podStartE2EDuration="2.52573826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:14:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:14:23.52506461 +0000 UTC m=+978.922256781" watchObservedRunningTime="2026-09-11 01:14:23.52573826 +0000 UTC m=+978.922930433"
Sep 11 01:14:23.541692 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:23.541647    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-6573d049-788d8d7cdf-v2zxr" podStartSLOduration=2.541634542 podStartE2EDuration="2.541634542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:14:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:14:23.540687659 +0000 UTC m=+978.937879832" watchObservedRunningTime="2026-09-11 01:14:23.541634542 +0000 UTC m=+978.938826714"
Sep 11 01:14:30.259441 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:30.259364    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf"]
Sep 11 01:14:30.261806 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:30.261789    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf"
Sep 11 01:14:30.265207 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:30.265180    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-e98d1e\"/\"default-dockercfg-42z99\""
Sep 11 01:14:30.265332 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:30.265213    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-e98d1e\"/\"openshift-service-ca.crt\""
Sep 11 01:14:30.265332 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:30.265260    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-e98d1e\"/\"kube-root-ca.crt\""
Sep 11 01:14:30.265470 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:30.265343    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-e98d1e\"/\"test-model-e98d1e-kserve-self-signed-certs\""
Sep 11 01:14:30.271666 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:30.271645    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf"]
Sep 11 01:14:30.370395 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:30.370359    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/effe0140-b5e3-40bc-856b-d620885a61a1-model-cache\") pod \"test-model-e98d1e-kserve-6454d8c4df-dwccf\" (UID: \"effe0140-b5e3-40bc-856b-d620885a61a1\") " pod="ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf"
Sep 11 01:14:30.370557 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:30.370404    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/effe0140-b5e3-40bc-856b-d620885a61a1-tls-certs\") pod \"test-model-e98d1e-kserve-6454d8c4df-dwccf\" (UID: \"effe0140-b5e3-40bc-856b-d620885a61a1\") " pod="ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf"
Sep 11 01:14:30.370557 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:30.370424    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/effe0140-b5e3-40bc-856b-d620885a61a1-tmp-dir\") pod \"test-model-e98d1e-kserve-6454d8c4df-dwccf\" (UID: \"effe0140-b5e3-40bc-856b-d620885a61a1\") " pod="ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf"
Sep 11 01:14:30.370557 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:30.370511    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2glqt\" (UniqueName: \"kubernetes.io/projected/effe0140-b5e3-40bc-856b-d620885a61a1-kube-api-access-2glqt\") pod \"test-model-e98d1e-kserve-6454d8c4df-dwccf\" (UID: \"effe0140-b5e3-40bc-856b-d620885a61a1\") " pod="ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf"
Sep 11 01:14:30.370669 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:30.370565    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/effe0140-b5e3-40bc-856b-d620885a61a1-home\") pod \"test-model-e98d1e-kserve-6454d8c4df-dwccf\" (UID: \"effe0140-b5e3-40bc-856b-d620885a61a1\") " pod="ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf"
Sep 11 01:14:30.370669 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:30.370630    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/effe0140-b5e3-40bc-856b-d620885a61a1-dshm\") pod \"test-model-e98d1e-kserve-6454d8c4df-dwccf\" (UID: \"effe0140-b5e3-40bc-856b-d620885a61a1\") " pod="ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf"
Sep 11 01:14:30.469120 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:30.469090    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:30.471297 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:30.471275    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/effe0140-b5e3-40bc-856b-d620885a61a1-model-cache\") pod \"test-model-e98d1e-kserve-6454d8c4df-dwccf\" (UID: \"effe0140-b5e3-40bc-856b-d620885a61a1\") " pod="ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf"
Sep 11 01:14:30.471414 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:30.471311    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/effe0140-b5e3-40bc-856b-d620885a61a1-tls-certs\") pod \"test-model-e98d1e-kserve-6454d8c4df-dwccf\" (UID: \"effe0140-b5e3-40bc-856b-d620885a61a1\") " pod="ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf"
Sep 11 01:14:30.471414 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:30.471335    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/effe0140-b5e3-40bc-856b-d620885a61a1-tmp-dir\") pod \"test-model-e98d1e-kserve-6454d8c4df-dwccf\" (UID: \"effe0140-b5e3-40bc-856b-d620885a61a1\") " pod="ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf"
Sep 11 01:14:30.471414 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:30.471400    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2glqt\" (UniqueName: \"kubernetes.io/projected/effe0140-b5e3-40bc-856b-d620885a61a1-kube-api-access-2glqt\") pod \"test-model-e98d1e-kserve-6454d8c4df-dwccf\" (UID: \"effe0140-b5e3-40bc-856b-d620885a61a1\") " pod="ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf"
Sep 11 01:14:30.471574 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:30.471461    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/effe0140-b5e3-40bc-856b-d620885a61a1-home\") pod \"test-model-e98d1e-kserve-6454d8c4df-dwccf\" (UID: \"effe0140-b5e3-40bc-856b-d620885a61a1\") " pod="ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf"
Sep 11 01:14:30.471574 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:30.471490    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/effe0140-b5e3-40bc-856b-d620885a61a1-dshm\") pod \"test-model-e98d1e-kserve-6454d8c4df-dwccf\" (UID: \"effe0140-b5e3-40bc-856b-d620885a61a1\") " pod="ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf"
Sep 11 01:14:30.471678 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:30.471649    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/effe0140-b5e3-40bc-856b-d620885a61a1-model-cache\") pod \"test-model-e98d1e-kserve-6454d8c4df-dwccf\" (UID: \"effe0140-b5e3-40bc-856b-d620885a61a1\") " pod="ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf"
Sep 11 01:14:30.471730 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:30.471677    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/effe0140-b5e3-40bc-856b-d620885a61a1-tmp-dir\") pod \"test-model-e98d1e-kserve-6454d8c4df-dwccf\" (UID: \"effe0140-b5e3-40bc-856b-d620885a61a1\") " pod="ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf"
Sep 11 01:14:30.471801 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:30.471740    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/effe0140-b5e3-40bc-856b-d620885a61a1-home\") pod \"test-model-e98d1e-kserve-6454d8c4df-dwccf\" (UID: \"effe0140-b5e3-40bc-856b-d620885a61a1\") " pod="ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf"
Sep 11 01:14:30.473809 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:30.473784    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/effe0140-b5e3-40bc-856b-d620885a61a1-dshm\") pod \"test-model-e98d1e-kserve-6454d8c4df-dwccf\" (UID: \"effe0140-b5e3-40bc-856b-d620885a61a1\") " pod="ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf"
Sep 11 01:14:30.474023 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:30.474008    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/effe0140-b5e3-40bc-856b-d620885a61a1-tls-certs\") pod \"test-model-e98d1e-kserve-6454d8c4df-dwccf\" (UID: \"effe0140-b5e3-40bc-856b-d620885a61a1\") " pod="ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf"
Sep 11 01:14:30.478694 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:30.478673    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2glqt\" (UniqueName: \"kubernetes.io/projected/effe0140-b5e3-40bc-856b-d620885a61a1-kube-api-access-2glqt\") pod \"test-model-e98d1e-kserve-6454d8c4df-dwccf\" (UID: \"effe0140-b5e3-40bc-856b-d620885a61a1\") " pod="ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf"
Sep 11 01:14:30.572488 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:30.572409    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf"
Sep 11 01:14:30.842220 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:30.842194    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf"]
Sep 11 01:14:30.844190 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:14:30.844165    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeffe0140_b5e3_40bc_856b_d620885a61a1.slice/crio-e6da470e2e3b5e84bacd54d049484ccd48d040a0f76ba9c9739972ba860a9445 WatchSource:0}: Error finding container e6da470e2e3b5e84bacd54d049484ccd48d040a0f76ba9c9739972ba860a9445: Status 404 returned error can't find the container with id e6da470e2e3b5e84bacd54d049484ccd48d040a0f76ba9c9739972ba860a9445
Sep 11 01:14:31.563649 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:31.563611    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf" event={"ID":"effe0140-b5e3-40bc-856b-d620885a61a1","Type":"ContainerStarted","Data":"f1407ab9b8d43fbb0fd863efb01f47ed665e9645ea8211f1d11c3138cf9a7d34"}
Sep 11 01:14:31.563649 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:31.563662    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf" event={"ID":"effe0140-b5e3-40bc-856b-d620885a61a1","Type":"ContainerStarted","Data":"e6da470e2e3b5e84bacd54d049484ccd48d040a0f76ba9c9739972ba860a9445"}
Sep 11 01:14:31.581360 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:31.581308    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf" podStartSLOduration=1.581292658 podStartE2EDuration="1.581292658s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:14:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:14:31.579581157 +0000 UTC m=+986.976773331" watchObservedRunningTime="2026-09-11 01:14:31.581292658 +0000 UTC m=+986.978484830"
Sep 11 01:14:34.521751 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:34.521711    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-6573d049-69dfd459bb-cmfnr"
Sep 11 01:14:34.522177 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:34.521781    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-6573d049-788d8d7cdf-v2zxr"
Sep 11 01:14:40.573501 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:40.573461    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf"
Sep 11 01:14:40.573987 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:40.573519    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf"
Sep 11 01:14:40.586384 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:40.586359    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf"
Sep 11 01:14:40.624817 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:40.624783    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf"
Sep 11 01:14:42.198855 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:42.198286    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-2c1e69-zl2gp"]
Sep 11 01:14:42.202331 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:42.202309    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-2c1e69-zl2gp"
Sep 11 01:14:42.207593 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:42.207387    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 01:14:42.208011 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:42.207993    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q6gbn\""
Sep 11 01:14:42.225497 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:42.225471    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-2c1e69-zl2gp"]
Sep 11 01:14:42.283732 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:42.283696    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b1f299b-72a8-45b0-ac9a-6352d062a94c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-2c1e69-zl2gp\" (UID: \"3b1f299b-72a8-45b0-ac9a-6352d062a94c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-2c1e69-zl2gp"
Sep 11 01:14:42.384826 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:42.384793    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b1f299b-72a8-45b0-ac9a-6352d062a94c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-2c1e69-zl2gp\" (UID: \"3b1f299b-72a8-45b0-ac9a-6352d062a94c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-2c1e69-zl2gp"
Sep 11 01:14:42.385444 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:42.385420    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b1f299b-72a8-45b0-ac9a-6352d062a94c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-2c1e69-zl2gp\" (UID: \"3b1f299b-72a8-45b0-ac9a-6352d062a94c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-2c1e69-zl2gp"
Sep 11 01:14:42.524086 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:42.523992    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-2c1e69-zl2gp"
Sep 11 01:14:42.796626 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:42.796597    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-2c1e69-zl2gp"]
Sep 11 01:14:42.799429 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:14:42.799402    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b1f299b_72a8_45b0_ac9a_6352d062a94c.slice/crio-47f309d8bc58f79e47f68a6b8a4a47d60d2c31ecdfcd4648fe8046ea0f06f4d4 WatchSource:0}: Error finding container 47f309d8bc58f79e47f68a6b8a4a47d60d2c31ecdfcd4648fe8046ea0f06f4d4: Status 404 returned error can't find the container with id 47f309d8bc58f79e47f68a6b8a4a47d60d2c31ecdfcd4648fe8046ea0f06f4d4
Sep 11 01:14:42.894595 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:42.894564    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:43.629479 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:43.629442    2574 generic.go:356] "Generic (PLEG): container finished" podID="3b1f299b-72a8-45b0-ac9a-6352d062a94c" containerID="fa23161924d23c7bf0cdf2fef18cf161f1d491389e406605121db74c0048b856" exitCode=0
Sep 11 01:14:43.629903 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:43.629531    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-2c1e69-zl2gp" event={"ID":"3b1f299b-72a8-45b0-ac9a-6352d062a94c","Type":"ContainerDied","Data":"fa23161924d23c7bf0cdf2fef18cf161f1d491389e406605121db74c0048b856"}
Sep 11 01:14:43.629903 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:43.629627    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-2c1e69-zl2gp" event={"ID":"3b1f299b-72a8-45b0-ac9a-6352d062a94c","Type":"ContainerStarted","Data":"47f309d8bc58f79e47f68a6b8a4a47d60d2c31ecdfcd4648fe8046ea0f06f4d4"}
Sep 11 01:14:43.731352 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:43.731260    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:44.772968 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:44.772943    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-2c1e69-zl2gp"
Sep 11 01:14:44.909427 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:44.909396    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b1f299b-72a8-45b0-ac9a-6352d062a94c-maas-api-service-ca\") pod \"3b1f299b-72a8-45b0-ac9a-6352d062a94c\" (UID: \"3b1f299b-72a8-45b0-ac9a-6352d062a94c\") "
Sep 11 01:14:44.909780 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:44.909757    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b1f299b-72a8-45b0-ac9a-6352d062a94c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3b1f299b-72a8-45b0-ac9a-6352d062a94c" (UID: "3b1f299b-72a8-45b0-ac9a-6352d062a94c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:14:45.010768 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:45.010729    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b1f299b-72a8-45b0-ac9a-6352d062a94c-maas-api-service-ca\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:14:45.078957 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:45.078903    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6573d049-24jwh"]
Sep 11 01:14:45.079396 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:45.079378    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b1f299b-72a8-45b0-ac9a-6352d062a94c" containerName="revoke-keys"
Sep 11 01:14:45.079396 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:45.079397    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b1f299b-72a8-45b0-ac9a-6352d062a94c" containerName="revoke-keys"
Sep 11 01:14:45.079549 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:45.079476    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b1f299b-72a8-45b0-ac9a-6352d062a94c" containerName="revoke-keys"
Sep 11 01:14:45.082701 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:45.082680    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6573d049-24jwh"
Sep 11 01:14:45.089216 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:45.089152    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6573d049-24jwh"]
Sep 11 01:14:45.212626 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:45.212536    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7564a16f-aa28-4a08-91bb-5c05477f3a9b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-6573d049-24jwh\" (UID: \"7564a16f-aa28-4a08-91bb-5c05477f3a9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6573d049-24jwh"
Sep 11 01:14:45.314020 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:45.313984    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7564a16f-aa28-4a08-91bb-5c05477f3a9b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-6573d049-24jwh\" (UID: \"7564a16f-aa28-4a08-91bb-5c05477f3a9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6573d049-24jwh"
Sep 11 01:14:45.314616 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:45.314595    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7564a16f-aa28-4a08-91bb-5c05477f3a9b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-6573d049-24jwh\" (UID: \"7564a16f-aa28-4a08-91bb-5c05477f3a9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6573d049-24jwh"
Sep 11 01:14:45.395077 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:45.395049    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6573d049-24jwh"
Sep 11 01:14:45.640981 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:45.640889    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-2c1e69-zl2gp"
Sep 11 01:14:45.640981 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:45.640934    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-2c1e69-zl2gp" event={"ID":"3b1f299b-72a8-45b0-ac9a-6352d062a94c","Type":"ContainerDied","Data":"47f309d8bc58f79e47f68a6b8a4a47d60d2c31ecdfcd4648fe8046ea0f06f4d4"}
Sep 11 01:14:45.640981 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:45.640979    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="47f309d8bc58f79e47f68a6b8a4a47d60d2c31ecdfcd4648fe8046ea0f06f4d4"
Sep 11 01:14:45.663599 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:45.663575    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6573d049-24jwh"]
Sep 11 01:14:45.665579 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:14:45.665552    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7564a16f_aa28_4a08_91bb_5c05477f3a9b.slice/crio-493d2c75fed381705ab55ff7c04a30b2e961d80c59f30b2f8807c610b8f0810d WatchSource:0}: Error finding container 493d2c75fed381705ab55ff7c04a30b2e961d80c59f30b2f8807c610b8f0810d: Status 404 returned error can't find the container with id 493d2c75fed381705ab55ff7c04a30b2e961d80c59f30b2f8807c610b8f0810d
Sep 11 01:14:46.649381 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:46.649289    2574 generic.go:356] "Generic (PLEG): container finished" podID="7564a16f-aa28-4a08-91bb-5c05477f3a9b" containerID="add88a5f21591f3a0ae63a12349b0b30e565262bd890ca9e5d42857e2b4282e2" exitCode=0
Sep 11 01:14:46.649859 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:46.649396    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6573d049-24jwh" event={"ID":"7564a16f-aa28-4a08-91bb-5c05477f3a9b","Type":"ContainerDied","Data":"add88a5f21591f3a0ae63a12349b0b30e565262bd890ca9e5d42857e2b4282e2"}
Sep 11 01:14:46.649859 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:46.649444    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6573d049-24jwh" event={"ID":"7564a16f-aa28-4a08-91bb-5c05477f3a9b","Type":"ContainerStarted","Data":"493d2c75fed381705ab55ff7c04a30b2e961d80c59f30b2f8807c610b8f0810d"}
Sep 11 01:14:46.673181 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:46.673136    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:47.787238 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:47.787208    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6573d049-24jwh"
Sep 11 01:14:47.939717 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:47.939684    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7564a16f-aa28-4a08-91bb-5c05477f3a9b-maas-api-service-ca\") pod \"7564a16f-aa28-4a08-91bb-5c05477f3a9b\" (UID: \"7564a16f-aa28-4a08-91bb-5c05477f3a9b\") "
Sep 11 01:14:47.940041 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:47.940019    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7564a16f-aa28-4a08-91bb-5c05477f3a9b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7564a16f-aa28-4a08-91bb-5c05477f3a9b" (UID: "7564a16f-aa28-4a08-91bb-5c05477f3a9b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:14:48.040968 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:48.040905    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7564a16f-aa28-4a08-91bb-5c05477f3a9b-maas-api-service-ca\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:14:48.660567 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:48.660535    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6573d049-24jwh" event={"ID":"7564a16f-aa28-4a08-91bb-5c05477f3a9b","Type":"ContainerDied","Data":"493d2c75fed381705ab55ff7c04a30b2e961d80c59f30b2f8807c610b8f0810d"}
Sep 11 01:14:48.660567 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:48.660564    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6573d049-24jwh"
Sep 11 01:14:48.660778 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:48.660583    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="493d2c75fed381705ab55ff7c04a30b2e961d80c59f30b2f8807c610b8f0810d"
Sep 11 01:14:51.842695 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:51.842665    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:51.873492 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:51.873458    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-6573d049-788d8d7cdf-v2zxr"]
Sep 11 01:14:51.873716 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:51.873690    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-6573d049-788d8d7cdf-v2zxr" podUID="a38c1497-facc-4a2b-93d3-fa7d1dc28f50" containerName="payload-processing" containerID="cri-o://7596ba725bc826cec5c0c1c06b6e9079ef4418679454e67eeaf691fc1fdef8c1" gracePeriod=30
Sep 11 01:14:51.879088 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:51.879051    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6573d049-69dfd459bb-cmfnr"]
Sep 11 01:14:51.879296 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:51.879270    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-6573d049-69dfd459bb-cmfnr" podUID="db358587-cd47-4b7c-8667-8641b2af5334" containerName="payload-pre-processing" containerID="cri-o://30e6aa53e5e7066dee2e87e1181220314146a32f60904ed6b0deef5de97cfef0" gracePeriod=30
Sep 11 01:14:51.912595 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:51.912570    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:52.011815 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.011752    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:52.037604 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.037525    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6573d049-24jwh"]
Sep 11 01:14:52.039651 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.039623    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6573d049-24jwh"]
Sep 11 01:14:52.069653 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.069628    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:52.156507 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.156482    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-6573d049-788d8d7cdf-v2zxr"
Sep 11 01:14:52.160691 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.160660    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-6573d049-69dfd459bb-cmfnr"
Sep 11 01:14:52.187270 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.187234    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:52.277361 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.277323    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lrgp\" (UniqueName: \"kubernetes.io/projected/a38c1497-facc-4a2b-93d3-fa7d1dc28f50-kube-api-access-9lrgp\") pod \"a38c1497-facc-4a2b-93d3-fa7d1dc28f50\" (UID: \"a38c1497-facc-4a2b-93d3-fa7d1dc28f50\") "
Sep 11 01:14:52.277571 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.277401    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a38c1497-facc-4a2b-93d3-fa7d1dc28f50-plugins-config-volume\") pod \"a38c1497-facc-4a2b-93d3-fa7d1dc28f50\" (UID: \"a38c1497-facc-4a2b-93d3-fa7d1dc28f50\") "
Sep 11 01:14:52.277571 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.277434    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jn7nz\" (UniqueName: \"kubernetes.io/projected/db358587-cd47-4b7c-8667-8641b2af5334-kube-api-access-jn7nz\") pod \"db358587-cd47-4b7c-8667-8641b2af5334\" (UID: \"db358587-cd47-4b7c-8667-8641b2af5334\") "
Sep 11 01:14:52.277571 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.277475    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db358587-cd47-4b7c-8667-8641b2af5334-plugins-config-volume\") pod \"db358587-cd47-4b7c-8667-8641b2af5334\" (UID: \"db358587-cd47-4b7c-8667-8641b2af5334\") "
Sep 11 01:14:52.277838 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.277811    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a38c1497-facc-4a2b-93d3-fa7d1dc28f50-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a38c1497-facc-4a2b-93d3-fa7d1dc28f50" (UID: "a38c1497-facc-4a2b-93d3-fa7d1dc28f50"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:14:52.277953 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.277902    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db358587-cd47-4b7c-8667-8641b2af5334-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "db358587-cd47-4b7c-8667-8641b2af5334" (UID: "db358587-cd47-4b7c-8667-8641b2af5334"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:14:52.279684 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.279664    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a38c1497-facc-4a2b-93d3-fa7d1dc28f50-kube-api-access-9lrgp" (OuterVolumeSpecName: "kube-api-access-9lrgp") pod "a38c1497-facc-4a2b-93d3-fa7d1dc28f50" (UID: "a38c1497-facc-4a2b-93d3-fa7d1dc28f50"). InnerVolumeSpecName "kube-api-access-9lrgp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:14:52.279766 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.279687    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db358587-cd47-4b7c-8667-8641b2af5334-kube-api-access-jn7nz" (OuterVolumeSpecName: "kube-api-access-jn7nz") pod "db358587-cd47-4b7c-8667-8641b2af5334" (UID: "db358587-cd47-4b7c-8667-8641b2af5334"). InnerVolumeSpecName "kube-api-access-jn7nz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:14:52.379182 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.379090    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lrgp\" (UniqueName: \"kubernetes.io/projected/a38c1497-facc-4a2b-93d3-fa7d1dc28f50-kube-api-access-9lrgp\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:14:52.379182 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.379125    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a38c1497-facc-4a2b-93d3-fa7d1dc28f50-plugins-config-volume\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:14:52.379182 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.379137    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jn7nz\" (UniqueName: \"kubernetes.io/projected/db358587-cd47-4b7c-8667-8641b2af5334-kube-api-access-jn7nz\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:14:52.379182 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.379146    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db358587-cd47-4b7c-8667-8641b2af5334-plugins-config-volume\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:14:52.436354 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.436323    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:52.680800 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.680762    2574 generic.go:356] "Generic (PLEG): container finished" podID="db358587-cd47-4b7c-8667-8641b2af5334" containerID="30e6aa53e5e7066dee2e87e1181220314146a32f60904ed6b0deef5de97cfef0" exitCode=0
Sep 11 01:14:52.681086 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.680809    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-6573d049-69dfd459bb-cmfnr"
Sep 11 01:14:52.681086 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.680852    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-6573d049-69dfd459bb-cmfnr" event={"ID":"db358587-cd47-4b7c-8667-8641b2af5334","Type":"ContainerDied","Data":"30e6aa53e5e7066dee2e87e1181220314146a32f60904ed6b0deef5de97cfef0"}
Sep 11 01:14:52.681086 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.680904    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-6573d049-69dfd459bb-cmfnr" event={"ID":"db358587-cd47-4b7c-8667-8641b2af5334","Type":"ContainerDied","Data":"5f246e1df4d8542d7cb43c2449c1742c8f2f00d1e14b6b9edefb3136e9ea311f"}
Sep 11 01:14:52.681086 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.680940    2574 scope.go:117] "RemoveContainer" containerID="30e6aa53e5e7066dee2e87e1181220314146a32f60904ed6b0deef5de97cfef0"
Sep 11 01:14:52.682471 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.682446    2574 generic.go:356] "Generic (PLEG): container finished" podID="a38c1497-facc-4a2b-93d3-fa7d1dc28f50" containerID="7596ba725bc826cec5c0c1c06b6e9079ef4418679454e67eeaf691fc1fdef8c1" exitCode=0
Sep 11 01:14:52.682581 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.682512    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-6573d049-788d8d7cdf-v2zxr" event={"ID":"a38c1497-facc-4a2b-93d3-fa7d1dc28f50","Type":"ContainerDied","Data":"7596ba725bc826cec5c0c1c06b6e9079ef4418679454e67eeaf691fc1fdef8c1"}
Sep 11 01:14:52.682581 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.682547    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-6573d049-788d8d7cdf-v2zxr" event={"ID":"a38c1497-facc-4a2b-93d3-fa7d1dc28f50","Type":"ContainerDied","Data":"e0cf31e324ac66ea04d50ca5e2830b7a9a9f47e42d0f9d08816042e9df637e05"}
Sep 11 01:14:52.682663 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.682568    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-6573d049-788d8d7cdf-v2zxr"
Sep 11 01:14:52.691638 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.691620    2574 scope.go:117] "RemoveContainer" containerID="30e6aa53e5e7066dee2e87e1181220314146a32f60904ed6b0deef5de97cfef0"
Sep 11 01:14:52.692036 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:14:52.692016    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30e6aa53e5e7066dee2e87e1181220314146a32f60904ed6b0deef5de97cfef0\": container with ID starting with 30e6aa53e5e7066dee2e87e1181220314146a32f60904ed6b0deef5de97cfef0 not found: ID does not exist" containerID="30e6aa53e5e7066dee2e87e1181220314146a32f60904ed6b0deef5de97cfef0"
Sep 11 01:14:52.692137 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.692049    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30e6aa53e5e7066dee2e87e1181220314146a32f60904ed6b0deef5de97cfef0"} err="failed to get container status \"30e6aa53e5e7066dee2e87e1181220314146a32f60904ed6b0deef5de97cfef0\": rpc error: code = NotFound desc = could not find container \"30e6aa53e5e7066dee2e87e1181220314146a32f60904ed6b0deef5de97cfef0\": container with ID starting with 30e6aa53e5e7066dee2e87e1181220314146a32f60904ed6b0deef5de97cfef0 not found: ID does not exist"
Sep 11 01:14:52.692137 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.692072    2574 scope.go:117] "RemoveContainer" containerID="7596ba725bc826cec5c0c1c06b6e9079ef4418679454e67eeaf691fc1fdef8c1"
Sep 11 01:14:52.703471 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.702945    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6573d049-69dfd459bb-cmfnr"]
Sep 11 01:14:52.703555 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.703516    2574 scope.go:117] "RemoveContainer" containerID="7596ba725bc826cec5c0c1c06b6e9079ef4418679454e67eeaf691fc1fdef8c1"
Sep 11 01:14:52.704221 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:14:52.704198    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7596ba725bc826cec5c0c1c06b6e9079ef4418679454e67eeaf691fc1fdef8c1\": container with ID starting with 7596ba725bc826cec5c0c1c06b6e9079ef4418679454e67eeaf691fc1fdef8c1 not found: ID does not exist" containerID="7596ba725bc826cec5c0c1c06b6e9079ef4418679454e67eeaf691fc1fdef8c1"
Sep 11 01:14:52.704305 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.704226    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7596ba725bc826cec5c0c1c06b6e9079ef4418679454e67eeaf691fc1fdef8c1"} err="failed to get container status \"7596ba725bc826cec5c0c1c06b6e9079ef4418679454e67eeaf691fc1fdef8c1\": rpc error: code = NotFound desc = could not find container \"7596ba725bc826cec5c0c1c06b6e9079ef4418679454e67eeaf691fc1fdef8c1\": container with ID starting with 7596ba725bc826cec5c0c1c06b6e9079ef4418679454e67eeaf691fc1fdef8c1 not found: ID does not exist"
Sep 11 01:14:52.706017 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.706000    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6573d049-69dfd459bb-cmfnr"]
Sep 11 01:14:52.716477 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.716405    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-6573d049-788d8d7cdf-v2zxr"]
Sep 11 01:14:52.719077 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.719056    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-6573d049-788d8d7cdf-v2zxr"]
Sep 11 01:14:52.955640 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:52.955554    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:53.072960 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.072923    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:53.100065 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.100034    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7564a16f-aa28-4a08-91bb-5c05477f3a9b" path="/var/lib/kubelet/pods/7564a16f-aa28-4a08-91bb-5c05477f3a9b/volumes"
Sep 11 01:14:53.100400 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.100386    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a38c1497-facc-4a2b-93d3-fa7d1dc28f50" path="/var/lib/kubelet/pods/a38c1497-facc-4a2b-93d3-fa7d1dc28f50/volumes"
Sep 11 01:14:53.100697 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.100686    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db358587-cd47-4b7c-8667-8641b2af5334" path="/var/lib/kubelet/pods/db358587-cd47-4b7c-8667-8641b2af5334/volumes"
Sep 11 01:14:53.179810 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.179761    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:53.201465 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.201429    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-2c1e69-6ff7b679df-nvthr"]
Sep 11 01:14:53.201705 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.201676    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-2c1e69-6ff7b679df-nvthr" podUID="93b6d88b-42f3-44ae-b042-54e5f8e0f763" containerName="payload-processing" containerID="cri-o://87133ddf8820decfdc5c1bff4ecd5d6c1044a4d3d974e5d212a504e9b42ee787" gracePeriod=30
Sep 11 01:14:53.210795 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.210766    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2c1e69-6cd8cbffb6-k5pwp"]
Sep 11 01:14:53.211047 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.211013    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-2c1e69-6cd8cbffb6-k5pwp" podUID="af08e083-3649-4feb-8f36-2eb01da9a3db" containerName="payload-pre-processing" containerID="cri-o://6d4d8016a5e811e5b8cae85603c243aa8cf3ef9dec8bc9ecea836bee82193cc3" gracePeriod=30
Sep 11 01:14:53.249524 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.249497    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:53.323457 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.323427    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:53.401610 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.401571    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-2c1e69-zl2gp"]
Sep 11 01:14:53.406233 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.406199    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:53.406233 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.406229    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-2c1e69-zl2gp"]
Sep 11 01:14:53.462517 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.462458    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-2c1e69-6ff7b679df-nvthr"
Sep 11 01:14:53.468887 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.468836    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:53.485581 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.485558    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-2c1e69-6cd8cbffb6-k5pwp"
Sep 11 01:14:53.590595 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.590563    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kb5br\" (UniqueName: \"kubernetes.io/projected/af08e083-3649-4feb-8f36-2eb01da9a3db-kube-api-access-kb5br\") pod \"af08e083-3649-4feb-8f36-2eb01da9a3db\" (UID: \"af08e083-3649-4feb-8f36-2eb01da9a3db\") "
Sep 11 01:14:53.590595 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.590600    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af08e083-3649-4feb-8f36-2eb01da9a3db-plugins-config-volume\") pod \"af08e083-3649-4feb-8f36-2eb01da9a3db\" (UID: \"af08e083-3649-4feb-8f36-2eb01da9a3db\") "
Sep 11 01:14:53.590861 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.590630    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93b6d88b-42f3-44ae-b042-54e5f8e0f763-plugins-config-volume\") pod \"93b6d88b-42f3-44ae-b042-54e5f8e0f763\" (UID: \"93b6d88b-42f3-44ae-b042-54e5f8e0f763\") "
Sep 11 01:14:53.590861 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.590661    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-swbb6\" (UniqueName: \"kubernetes.io/projected/93b6d88b-42f3-44ae-b042-54e5f8e0f763-kube-api-access-swbb6\") pod \"93b6d88b-42f3-44ae-b042-54e5f8e0f763\" (UID: \"93b6d88b-42f3-44ae-b042-54e5f8e0f763\") "
Sep 11 01:14:53.591017 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.590993    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af08e083-3649-4feb-8f36-2eb01da9a3db-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "af08e083-3649-4feb-8f36-2eb01da9a3db" (UID: "af08e083-3649-4feb-8f36-2eb01da9a3db"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:14:53.591101 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.591076    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93b6d88b-42f3-44ae-b042-54e5f8e0f763-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "93b6d88b-42f3-44ae-b042-54e5f8e0f763" (UID: "93b6d88b-42f3-44ae-b042-54e5f8e0f763"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:14:53.592797 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.592778    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af08e083-3649-4feb-8f36-2eb01da9a3db-kube-api-access-kb5br" (OuterVolumeSpecName: "kube-api-access-kb5br") pod "af08e083-3649-4feb-8f36-2eb01da9a3db" (UID: "af08e083-3649-4feb-8f36-2eb01da9a3db"). InnerVolumeSpecName "kube-api-access-kb5br". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:14:53.593017 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.592996    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93b6d88b-42f3-44ae-b042-54e5f8e0f763-kube-api-access-swbb6" (OuterVolumeSpecName: "kube-api-access-swbb6") pod "93b6d88b-42f3-44ae-b042-54e5f8e0f763" (UID: "93b6d88b-42f3-44ae-b042-54e5f8e0f763"). InnerVolumeSpecName "kube-api-access-swbb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:14:53.690014 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.689981    2574 generic.go:356] "Generic (PLEG): container finished" podID="93b6d88b-42f3-44ae-b042-54e5f8e0f763" containerID="87133ddf8820decfdc5c1bff4ecd5d6c1044a4d3d974e5d212a504e9b42ee787" exitCode=0
Sep 11 01:14:53.690162 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.690030    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-2c1e69-6ff7b679df-nvthr"
Sep 11 01:14:53.690162 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.690028    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-2c1e69-6ff7b679df-nvthr" event={"ID":"93b6d88b-42f3-44ae-b042-54e5f8e0f763","Type":"ContainerDied","Data":"87133ddf8820decfdc5c1bff4ecd5d6c1044a4d3d974e5d212a504e9b42ee787"}
Sep 11 01:14:53.690162 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.690081    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-2c1e69-6ff7b679df-nvthr" event={"ID":"93b6d88b-42f3-44ae-b042-54e5f8e0f763","Type":"ContainerDied","Data":"b53732f8a86205725a9e3f208203f454d803beb46a7ca572584f779ab652007f"}
Sep 11 01:14:53.690162 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.690104    2574 scope.go:117] "RemoveContainer" containerID="87133ddf8820decfdc5c1bff4ecd5d6c1044a4d3d974e5d212a504e9b42ee787"
Sep 11 01:14:53.691374 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.691351    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kb5br\" (UniqueName: \"kubernetes.io/projected/af08e083-3649-4feb-8f36-2eb01da9a3db-kube-api-access-kb5br\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:14:53.691374 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.691372    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af08e083-3649-4feb-8f36-2eb01da9a3db-plugins-config-volume\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:14:53.691573 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.691383    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93b6d88b-42f3-44ae-b042-54e5f8e0f763-plugins-config-volume\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:14:53.691573 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.691391    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-swbb6\" (UniqueName: \"kubernetes.io/projected/93b6d88b-42f3-44ae-b042-54e5f8e0f763-kube-api-access-swbb6\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:14:53.692566 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.692547    2574 generic.go:356] "Generic (PLEG): container finished" podID="af08e083-3649-4feb-8f36-2eb01da9a3db" containerID="6d4d8016a5e811e5b8cae85603c243aa8cf3ef9dec8bc9ecea836bee82193cc3" exitCode=0
Sep 11 01:14:53.692692 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.692579    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-2c1e69-6cd8cbffb6-k5pwp" event={"ID":"af08e083-3649-4feb-8f36-2eb01da9a3db","Type":"ContainerDied","Data":"6d4d8016a5e811e5b8cae85603c243aa8cf3ef9dec8bc9ecea836bee82193cc3"}
Sep 11 01:14:53.692692 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.692601    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-2c1e69-6cd8cbffb6-k5pwp"
Sep 11 01:14:53.692692 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.692611    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-2c1e69-6cd8cbffb6-k5pwp" event={"ID":"af08e083-3649-4feb-8f36-2eb01da9a3db","Type":"ContainerDied","Data":"81b4f477d78d11198d53ee903fc0434c8981c9469aef335d41227a18f5adc871"}
Sep 11 01:14:53.700803 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.700786    2574 scope.go:117] "RemoveContainer" containerID="87133ddf8820decfdc5c1bff4ecd5d6c1044a4d3d974e5d212a504e9b42ee787"
Sep 11 01:14:53.701095 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:14:53.701073    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87133ddf8820decfdc5c1bff4ecd5d6c1044a4d3d974e5d212a504e9b42ee787\": container with ID starting with 87133ddf8820decfdc5c1bff4ecd5d6c1044a4d3d974e5d212a504e9b42ee787 not found: ID does not exist" containerID="87133ddf8820decfdc5c1bff4ecd5d6c1044a4d3d974e5d212a504e9b42ee787"
Sep 11 01:14:53.701163 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.701102    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87133ddf8820decfdc5c1bff4ecd5d6c1044a4d3d974e5d212a504e9b42ee787"} err="failed to get container status \"87133ddf8820decfdc5c1bff4ecd5d6c1044a4d3d974e5d212a504e9b42ee787\": rpc error: code = NotFound desc = could not find container \"87133ddf8820decfdc5c1bff4ecd5d6c1044a4d3d974e5d212a504e9b42ee787\": container with ID starting with 87133ddf8820decfdc5c1bff4ecd5d6c1044a4d3d974e5d212a504e9b42ee787 not found: ID does not exist"
Sep 11 01:14:53.701163 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.701119    2574 scope.go:117] "RemoveContainer" containerID="6d4d8016a5e811e5b8cae85603c243aa8cf3ef9dec8bc9ecea836bee82193cc3"
Sep 11 01:14:53.708964 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.708945    2574 scope.go:117] "RemoveContainer" containerID="6d4d8016a5e811e5b8cae85603c243aa8cf3ef9dec8bc9ecea836bee82193cc3"
Sep 11 01:14:53.709241 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:14:53.709220    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d4d8016a5e811e5b8cae85603c243aa8cf3ef9dec8bc9ecea836bee82193cc3\": container with ID starting with 6d4d8016a5e811e5b8cae85603c243aa8cf3ef9dec8bc9ecea836bee82193cc3 not found: ID does not exist" containerID="6d4d8016a5e811e5b8cae85603c243aa8cf3ef9dec8bc9ecea836bee82193cc3"
Sep 11 01:14:53.709311 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.709249    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d4d8016a5e811e5b8cae85603c243aa8cf3ef9dec8bc9ecea836bee82193cc3"} err="failed to get container status \"6d4d8016a5e811e5b8cae85603c243aa8cf3ef9dec8bc9ecea836bee82193cc3\": rpc error: code = NotFound desc = could not find container \"6d4d8016a5e811e5b8cae85603c243aa8cf3ef9dec8bc9ecea836bee82193cc3\": container with ID starting with 6d4d8016a5e811e5b8cae85603c243aa8cf3ef9dec8bc9ecea836bee82193cc3 not found: ID does not exist"
Sep 11 01:14:53.715399 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.715352    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-2c1e69-6ff7b679df-nvthr"]
Sep 11 01:14:53.719558 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.719536    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-2c1e69-6ff7b679df-nvthr"]
Sep 11 01:14:53.732522 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.732496    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2c1e69-6cd8cbffb6-k5pwp"]
Sep 11 01:14:53.736533 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.736514    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2c1e69-6cd8cbffb6-k5pwp"]
Sep 11 01:14:53.770663 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:53.770639    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:54.516131 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:54.516098    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:14:55.100083 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:55.100050    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b1f299b-72a8-45b0-ac9a-6352d062a94c" path="/var/lib/kubelet/pods/3b1f299b-72a8-45b0-ac9a-6352d062a94c/volumes"
Sep 11 01:14:55.100450 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:55.100434    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93b6d88b-42f3-44ae-b042-54e5f8e0f763" path="/var/lib/kubelet/pods/93b6d88b-42f3-44ae-b042-54e5f8e0f763/volumes"
Sep 11 01:14:55.100819 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:55.100789    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af08e083-3649-4feb-8f36-2eb01da9a3db" path="/var/lib/kubelet/pods/af08e083-3649-4feb-8f36-2eb01da9a3db/volumes"
Sep 11 01:14:58.846173 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:14:58.846140    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:15:00.216743 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.216709    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-10a215-29818155-6mcdd"]
Sep 11 01:15:00.217321 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.217277    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a38c1497-facc-4a2b-93d3-fa7d1dc28f50" containerName="payload-processing"
Sep 11 01:15:00.217365 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.217326    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a38c1497-facc-4a2b-93d3-fa7d1dc28f50" containerName="payload-processing"
Sep 11 01:15:00.217365 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.217343    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af08e083-3649-4feb-8f36-2eb01da9a3db" containerName="payload-pre-processing"
Sep 11 01:15:00.217365 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.217352    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="af08e083-3649-4feb-8f36-2eb01da9a3db" containerName="payload-pre-processing"
Sep 11 01:15:00.217468 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.217372    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db358587-cd47-4b7c-8667-8641b2af5334" containerName="payload-pre-processing"
Sep 11 01:15:00.217468 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.217381    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="db358587-cd47-4b7c-8667-8641b2af5334" containerName="payload-pre-processing"
Sep 11 01:15:00.217468 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.217398    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93b6d88b-42f3-44ae-b042-54e5f8e0f763" containerName="payload-processing"
Sep 11 01:15:00.217468 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.217406    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="93b6d88b-42f3-44ae-b042-54e5f8e0f763" containerName="payload-processing"
Sep 11 01:15:00.217468 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.217416    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7564a16f-aa28-4a08-91bb-5c05477f3a9b" containerName="revoke-keys"
Sep 11 01:15:00.217468 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.217424    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7564a16f-aa28-4a08-91bb-5c05477f3a9b" containerName="revoke-keys"
Sep 11 01:15:00.217651 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.217518    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="93b6d88b-42f3-44ae-b042-54e5f8e0f763" containerName="payload-processing"
Sep 11 01:15:00.217651 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.217528    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="db358587-cd47-4b7c-8667-8641b2af5334" containerName="payload-pre-processing"
Sep 11 01:15:00.217651 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.217540    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="af08e083-3649-4feb-8f36-2eb01da9a3db" containerName="payload-pre-processing"
Sep 11 01:15:00.217651 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.217551    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7564a16f-aa28-4a08-91bb-5c05477f3a9b" containerName="revoke-keys"
Sep 11 01:15:00.217651 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.217561    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a38c1497-facc-4a2b-93d3-fa7d1dc28f50" containerName="payload-processing"
Sep 11 01:15:00.222441 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.222421    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f4de10-29818155-7n5jw"]
Sep 11 01:15:00.222560 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.222534    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-10a215-29818155-6mcdd"
Sep 11 01:15:00.226216 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.226198    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f4de10-29818155-7n5jw"
Sep 11 01:15:00.228673 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.228650    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q6gbn\""
Sep 11 01:15:00.232934 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.232890    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cd176e-29818155-9gr52"]
Sep 11 01:15:00.234303 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.234288    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-10a215-29818155-6mcdd"
Sep 11 01:15:00.236007 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.235985    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cd176e-29818155-9gr52"
Sep 11 01:15:00.241996 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.241978    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f4de10-29818155-7n5jw"
Sep 11 01:15:00.252972 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.252757    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-10a215-29818155-6mcdd"]
Sep 11 01:15:00.259603 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.259383    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f4de10-29818155-7n5jw"]
Sep 11 01:15:00.259603 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.259473    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cd176e-29818155-9gr52"]
Sep 11 01:15:00.285388 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.285198    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cd176e-29818155-9gr52"
Sep 11 01:15:00.525995 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.525969    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-10a215-29818155-6mcdd"]
Sep 11 01:15:00.528197 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:15:00.528173    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a48094a_84dd_4058_b886_f7a2fd9111b9.slice/crio-ce57e556c21cf5fff0dd3e5f84fc8dbe3c8a93893c2eccb7e9d754360054e929 WatchSource:0}: Error finding container ce57e556c21cf5fff0dd3e5f84fc8dbe3c8a93893c2eccb7e9d754360054e929: Status 404 returned error can't find the container with id ce57e556c21cf5fff0dd3e5f84fc8dbe3c8a93893c2eccb7e9d754360054e929
Sep 11 01:15:00.546719 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.546699    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f4de10-29818155-7n5jw"]
Sep 11 01:15:00.548315 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:15:00.548291    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a4144ca_0da8_4af7_b7d4_076d8bf18d8e.slice/crio-674e363bdf03cb8130b8184b45d3d4b6770f11241a64226696336d0975be40a7 WatchSource:0}: Error finding container 674e363bdf03cb8130b8184b45d3d4b6770f11241a64226696336d0975be40a7: Status 404 returned error can't find the container with id 674e363bdf03cb8130b8184b45d3d4b6770f11241a64226696336d0975be40a7
Sep 11 01:15:00.589330 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.589302    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cd176e-29818155-9gr52"]
Sep 11 01:15:00.591542 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:15:00.591519    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60473175_dd74_4ba5_8761_2c662884df97.slice/crio-a10320f9ba81ea9f6131005bbe8ace4ffe7615551690737fb0b8332bbff144aa WatchSource:0}: Error finding container a10320f9ba81ea9f6131005bbe8ace4ffe7615551690737fb0b8332bbff144aa: Status 404 returned error can't find the container with id a10320f9ba81ea9f6131005bbe8ace4ffe7615551690737fb0b8332bbff144aa
Sep 11 01:15:00.704004 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.703973    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:15:00.737576 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.737539    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cd176e-29818155-9gr52" event={"ID":"60473175-dd74-4ba5-8761-2c662884df97","Type":"ContainerStarted","Data":"a10320f9ba81ea9f6131005bbe8ace4ffe7615551690737fb0b8332bbff144aa"}
Sep 11 01:15:00.738829 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.738802    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-10a215-29818155-6mcdd" event={"ID":"7a48094a-84dd-4058-b886-f7a2fd9111b9","Type":"ContainerStarted","Data":"ce57e556c21cf5fff0dd3e5f84fc8dbe3c8a93893c2eccb7e9d754360054e929"}
Sep 11 01:15:00.739923 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.739888    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f4de10-29818155-7n5jw" event={"ID":"6a4144ca-0da8-4af7-b7d4-076d8bf18d8e","Type":"ContainerStarted","Data":"674e363bdf03cb8130b8184b45d3d4b6770f11241a64226696336d0975be40a7"}
Sep 11 01:15:00.779155 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.779097    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:15:00.864460 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.864430    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:15:00.960135 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:00.960101    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:15:01.440068 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:01.440036    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:15:01.752880 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:01.752844    2574 generic.go:356] "Generic (PLEG): container finished" podID="6a4144ca-0da8-4af7-b7d4-076d8bf18d8e" containerID="8a01247d2390af21b812ee0740e4cf6fb80482d3a2ab5bcb4787f9bbd8d98910" exitCode=0
Sep 11 01:15:01.753087 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:01.752947    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f4de10-29818155-7n5jw" event={"ID":"6a4144ca-0da8-4af7-b7d4-076d8bf18d8e","Type":"ContainerDied","Data":"8a01247d2390af21b812ee0740e4cf6fb80482d3a2ab5bcb4787f9bbd8d98910"}
Sep 11 01:15:01.754471 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:01.754445    2574 generic.go:356] "Generic (PLEG): container finished" podID="60473175-dd74-4ba5-8761-2c662884df97" containerID="24d1603ef3528d23f1130776a376fc9b78452dbca43889e0d4eb9776845c0024" exitCode=0
Sep 11 01:15:01.754584 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:01.754507    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cd176e-29818155-9gr52" event={"ID":"60473175-dd74-4ba5-8761-2c662884df97","Type":"ContainerDied","Data":"24d1603ef3528d23f1130776a376fc9b78452dbca43889e0d4eb9776845c0024"}
Sep 11 01:15:01.756069 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:01.756044    2574 generic.go:356] "Generic (PLEG): container finished" podID="7a48094a-84dd-4058-b886-f7a2fd9111b9" containerID="8e183fbd613ade268ad3ff12cb9dbe8b7948b604e47f14ea8ef0f4242842be1d" exitCode=0
Sep 11 01:15:01.756185 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:01.756127    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-10a215-29818155-6mcdd" event={"ID":"7a48094a-84dd-4058-b886-f7a2fd9111b9","Type":"ContainerDied","Data":"8e183fbd613ade268ad3ff12cb9dbe8b7948b604e47f14ea8ef0f4242842be1d"}
Sep 11 01:15:02.909996 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:02.909968    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f4de10-29818155-7n5jw"
Sep 11 01:15:02.959013 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:02.958988    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cd176e-29818155-9gr52"
Sep 11 01:15:02.962336 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:02.962310    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-10a215-29818155-6mcdd"
Sep 11 01:15:03.768656 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:03.768624    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f4de10-29818155-7n5jw"
Sep 11 01:15:03.768656 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:03.768643    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f4de10-29818155-7n5jw" event={"ID":"6a4144ca-0da8-4af7-b7d4-076d8bf18d8e","Type":"ContainerDied","Data":"674e363bdf03cb8130b8184b45d3d4b6770f11241a64226696336d0975be40a7"}
Sep 11 01:15:03.768877 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:03.768688    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="674e363bdf03cb8130b8184b45d3d4b6770f11241a64226696336d0975be40a7"
Sep 11 01:15:03.770010 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:03.769976    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cd176e-29818155-9gr52"
Sep 11 01:15:03.770161 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:03.769978    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cd176e-29818155-9gr52" event={"ID":"60473175-dd74-4ba5-8761-2c662884df97","Type":"ContainerDied","Data":"a10320f9ba81ea9f6131005bbe8ace4ffe7615551690737fb0b8332bbff144aa"}
Sep 11 01:15:03.770161 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:03.770110    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a10320f9ba81ea9f6131005bbe8ace4ffe7615551690737fb0b8332bbff144aa"
Sep 11 01:15:03.771580 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:03.771561    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-10a215-29818155-6mcdd"
Sep 11 01:15:03.771684 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:03.771580    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-10a215-29818155-6mcdd" event={"ID":"7a48094a-84dd-4058-b886-f7a2fd9111b9","Type":"ContainerDied","Data":"ce57e556c21cf5fff0dd3e5f84fc8dbe3c8a93893c2eccb7e9d754360054e929"}
Sep 11 01:15:03.771684 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:03.771628    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ce57e556c21cf5fff0dd3e5f84fc8dbe3c8a93893c2eccb7e9d754360054e929"
Sep 11 01:15:04.716888 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:04.716855    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:15:05.010329 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:05.010257    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:15:05.469500 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:05.469467    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:15:05.787197 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:05.787119    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:15:05.993468 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:05.993427    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:15:09.042523 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:09.042456    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:15:09.141661 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:09.141619    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:15:10.350870 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:10.350813    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:15:10.452572 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:10.452540    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:15:10.552960 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:10.552929    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:15:11.000418 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:11.000389    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:15:11.087923 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:11.087872    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:15:11.137407 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:11.137377    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-987d60-5d8998b46b-gbljd"]
Sep 11 01:15:11.137738 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:11.137726    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a48094a-84dd-4058-b886-f7a2fd9111b9" containerName="cleanup"
Sep 11 01:15:11.137790 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:11.137739    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a48094a-84dd-4058-b886-f7a2fd9111b9" containerName="cleanup"
Sep 11 01:15:11.137790 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:11.137762    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a4144ca-0da8-4af7-b7d4-076d8bf18d8e" containerName="cleanup"
Sep 11 01:15:11.137790 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:11.137767    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a4144ca-0da8-4af7-b7d4-076d8bf18d8e" containerName="cleanup"
Sep 11 01:15:11.137790 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:11.137773    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60473175-dd74-4ba5-8761-2c662884df97" containerName="cleanup"
Sep 11 01:15:11.137790 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:11.137778    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="60473175-dd74-4ba5-8761-2c662884df97" containerName="cleanup"
Sep 11 01:15:11.138009 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:11.137830    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="60473175-dd74-4ba5-8761-2c662884df97" containerName="cleanup"
Sep 11 01:15:11.138009 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:11.137840    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a4144ca-0da8-4af7-b7d4-076d8bf18d8e" containerName="cleanup"
Sep 11 01:15:11.138009 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:11.137846    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a48094a-84dd-4058-b886-f7a2fd9111b9" containerName="cleanup"
Sep 11 01:15:11.142515 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:11.142497    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-987d60-5d8998b46b-gbljd"
Sep 11 01:15:11.144855 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:11.144831    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-987d60\""
Sep 11 01:15:11.144952 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:11.144879    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-987d60-dockercfg-gchnz\""
Sep 11 01:15:11.150146 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:11.149871    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-987d60-5d8998b46b-gbljd"]
Sep 11 01:15:11.255148 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:11.255073    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6ce8479-aa9a-4b41-9d16-5cc95567b933-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-987d60-5d8998b46b-gbljd\" (UID: \"a6ce8479-aa9a-4b41-9d16-5cc95567b933\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-987d60-5d8998b46b-gbljd"
Sep 11 01:15:11.255148 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:11.255125    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wbp2\" (UniqueName: \"kubernetes.io/projected/a6ce8479-aa9a-4b41-9d16-5cc95567b933-kube-api-access-2wbp2\") pod \"payload-pre-processing-e2e-explicit-987d60-5d8998b46b-gbljd\" (UID: \"a6ce8479-aa9a-4b41-9d16-5cc95567b933\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-987d60-5d8998b46b-gbljd"
Sep 11 01:15:11.356435 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:11.356399    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wbp2\" (UniqueName: \"kubernetes.io/projected/a6ce8479-aa9a-4b41-9d16-5cc95567b933-kube-api-access-2wbp2\") pod \"payload-pre-processing-e2e-explicit-987d60-5d8998b46b-gbljd\" (UID: \"a6ce8479-aa9a-4b41-9d16-5cc95567b933\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-987d60-5d8998b46b-gbljd"
Sep 11 01:15:11.356806 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:11.356532    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6ce8479-aa9a-4b41-9d16-5cc95567b933-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-987d60-5d8998b46b-gbljd\" (UID: \"a6ce8479-aa9a-4b41-9d16-5cc95567b933\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-987d60-5d8998b46b-gbljd"
Sep 11 01:15:11.357247 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:11.357227    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6ce8479-aa9a-4b41-9d16-5cc95567b933-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-987d60-5d8998b46b-gbljd\" (UID: \"a6ce8479-aa9a-4b41-9d16-5cc95567b933\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-987d60-5d8998b46b-gbljd"
Sep 11 01:15:11.365018 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:11.364998    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wbp2\" (UniqueName: \"kubernetes.io/projected/a6ce8479-aa9a-4b41-9d16-5cc95567b933-kube-api-access-2wbp2\") pod \"payload-pre-processing-e2e-explicit-987d60-5d8998b46b-gbljd\" (UID: \"a6ce8479-aa9a-4b41-9d16-5cc95567b933\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-987d60-5d8998b46b-gbljd"
Sep 11 01:15:11.454929 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:11.454879    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-987d60-5d8998b46b-gbljd"
Sep 11 01:15:11.724230 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:11.724205    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-987d60-5d8998b46b-gbljd"]
Sep 11 01:15:11.726807 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:15:11.726777    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6ce8479_aa9a_4b41_9d16_5cc95567b933.slice/crio-1829646d973ef5b8b9f2b8128368ab1a030476e0f4b62ad764c91492dbeffcef WatchSource:0}: Error finding container 1829646d973ef5b8b9f2b8128368ab1a030476e0f4b62ad764c91492dbeffcef: Status 404 returned error can't find the container with id 1829646d973ef5b8b9f2b8128368ab1a030476e0f4b62ad764c91492dbeffcef
Sep 11 01:15:11.814885 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:11.814852    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-987d60-5d8998b46b-gbljd" event={"ID":"a6ce8479-aa9a-4b41-9d16-5cc95567b933","Type":"ContainerStarted","Data":"388a03f27eb87ea0382d7351446950da05649aac8a6a88ce7af18e28c1726ca0"}
Sep 11 01:15:11.815060 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:11.814957    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-987d60-5d8998b46b-gbljd" event={"ID":"a6ce8479-aa9a-4b41-9d16-5cc95567b933","Type":"ContainerStarted","Data":"1829646d973ef5b8b9f2b8128368ab1a030476e0f4b62ad764c91492dbeffcef"}
Sep 11 01:15:11.815060 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:11.814991    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-987d60-5d8998b46b-gbljd"
Sep 11 01:15:14.049589 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:14.049552    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:15:22.148980 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:22.148943    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:15:22.822940 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:22.822874    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-987d60-5d8998b46b-gbljd"
Sep 11 01:15:22.840724 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:22.840666    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-987d60-5d8998b46b-gbljd" podStartSLOduration=11.840649923 podStartE2EDuration="11.840649923s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:15:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:15:11.830750355 +0000 UTC m=+1027.227942527" watchObservedRunningTime="2026-09-11 01:15:22.840649923 +0000 UTC m=+1038.237842097"
Sep 11 01:15:27.455203 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:27.455167    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:15:33.147565 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:33.147527    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:15:33.253367 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:33.253335    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:15:33.355700 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:33.355662    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:15:34.123175 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:34.123137    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:15:38.184037 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:38.184000    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt"]
Sep 11 01:15:38.189454 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:38.189420    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt"
Sep 11 01:15:38.193145 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:38.193101    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-f4de10\"/\"default-dockercfg-mg9vz\""
Sep 11 01:15:38.193402 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:38.193213    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-f4de10\"/\"premium-w0-f4de10-kserve-self-signed-certs\""
Sep 11 01:15:38.193575 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:38.193213    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-f4de10\"/\"openshift-service-ca.crt\""
Sep 11 01:15:38.193575 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:38.193258    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-f4de10\"/\"kube-root-ca.crt\""
Sep 11 01:15:38.194668 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:38.194650    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt"]
Sep 11 01:15:38.271854 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:38.271822    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:15:38.293118 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:38.293091    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f75a0b21-b351-4447-8774-e7b34f3fd258-tmp-dir\") pod \"premium-w0-f4de10-kserve-7787487fb8-cswlt\" (UID: \"f75a0b21-b351-4447-8774-e7b34f3fd258\") " pod="e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt"
Sep 11 01:15:38.293246 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:38.293125    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f75a0b21-b351-4447-8774-e7b34f3fd258-dshm\") pod \"premium-w0-f4de10-kserve-7787487fb8-cswlt\" (UID: \"f75a0b21-b351-4447-8774-e7b34f3fd258\") " pod="e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt"
Sep 11 01:15:38.293246 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:38.293149    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f75a0b21-b351-4447-8774-e7b34f3fd258-model-cache\") pod \"premium-w0-f4de10-kserve-7787487fb8-cswlt\" (UID: \"f75a0b21-b351-4447-8774-e7b34f3fd258\") " pod="e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt"
Sep 11 01:15:38.293367 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:38.293239    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd4nd\" (UniqueName: \"kubernetes.io/projected/f75a0b21-b351-4447-8774-e7b34f3fd258-kube-api-access-zd4nd\") pod \"premium-w0-f4de10-kserve-7787487fb8-cswlt\" (UID: \"f75a0b21-b351-4447-8774-e7b34f3fd258\") " pod="e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt"
Sep 11 01:15:38.293367 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:38.293294    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f75a0b21-b351-4447-8774-e7b34f3fd258-home\") pod \"premium-w0-f4de10-kserve-7787487fb8-cswlt\" (UID: \"f75a0b21-b351-4447-8774-e7b34f3fd258\") " pod="e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt"
Sep 11 01:15:38.293367 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:38.293321    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f75a0b21-b351-4447-8774-e7b34f3fd258-tls-certs\") pod \"premium-w0-f4de10-kserve-7787487fb8-cswlt\" (UID: \"f75a0b21-b351-4447-8774-e7b34f3fd258\") " pod="e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt"
Sep 11 01:15:38.394054 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:38.394029    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f75a0b21-b351-4447-8774-e7b34f3fd258-tmp-dir\") pod \"premium-w0-f4de10-kserve-7787487fb8-cswlt\" (UID: \"f75a0b21-b351-4447-8774-e7b34f3fd258\") " pod="e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt"
Sep 11 01:15:38.394167 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:38.394073    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f75a0b21-b351-4447-8774-e7b34f3fd258-dshm\") pod \"premium-w0-f4de10-kserve-7787487fb8-cswlt\" (UID: \"f75a0b21-b351-4447-8774-e7b34f3fd258\") " pod="e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt"
Sep 11 01:15:38.394167 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:38.394097    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f75a0b21-b351-4447-8774-e7b34f3fd258-model-cache\") pod \"premium-w0-f4de10-kserve-7787487fb8-cswlt\" (UID: \"f75a0b21-b351-4447-8774-e7b34f3fd258\") " pod="e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt"
Sep 11 01:15:38.394167 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:38.394127    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd4nd\" (UniqueName: \"kubernetes.io/projected/f75a0b21-b351-4447-8774-e7b34f3fd258-kube-api-access-zd4nd\") pod \"premium-w0-f4de10-kserve-7787487fb8-cswlt\" (UID: \"f75a0b21-b351-4447-8774-e7b34f3fd258\") " pod="e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt"
Sep 11 01:15:38.394167 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:38.394156    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f75a0b21-b351-4447-8774-e7b34f3fd258-home\") pod \"premium-w0-f4de10-kserve-7787487fb8-cswlt\" (UID: \"f75a0b21-b351-4447-8774-e7b34f3fd258\") " pod="e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt"
Sep 11 01:15:38.394383 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:38.394175    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f75a0b21-b351-4447-8774-e7b34f3fd258-tls-certs\") pod \"premium-w0-f4de10-kserve-7787487fb8-cswlt\" (UID: \"f75a0b21-b351-4447-8774-e7b34f3fd258\") " pod="e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt"
Sep 11 01:15:38.394475 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:38.394441    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f75a0b21-b351-4447-8774-e7b34f3fd258-tmp-dir\") pod \"premium-w0-f4de10-kserve-7787487fb8-cswlt\" (UID: \"f75a0b21-b351-4447-8774-e7b34f3fd258\") " pod="e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt"
Sep 11 01:15:38.394585 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:38.394565    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f75a0b21-b351-4447-8774-e7b34f3fd258-home\") pod \"premium-w0-f4de10-kserve-7787487fb8-cswlt\" (UID: \"f75a0b21-b351-4447-8774-e7b34f3fd258\") " pod="e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt"
Sep 11 01:15:38.394631 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:38.394569    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f75a0b21-b351-4447-8774-e7b34f3fd258-model-cache\") pod \"premium-w0-f4de10-kserve-7787487fb8-cswlt\" (UID: \"f75a0b21-b351-4447-8774-e7b34f3fd258\") " pod="e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt"
Sep 11 01:15:38.396404 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:38.396385    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f75a0b21-b351-4447-8774-e7b34f3fd258-dshm\") pod \"premium-w0-f4de10-kserve-7787487fb8-cswlt\" (UID: \"f75a0b21-b351-4447-8774-e7b34f3fd258\") " pod="e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt"
Sep 11 01:15:38.396707 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:38.396691    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f75a0b21-b351-4447-8774-e7b34f3fd258-tls-certs\") pod \"premium-w0-f4de10-kserve-7787487fb8-cswlt\" (UID: \"f75a0b21-b351-4447-8774-e7b34f3fd258\") " pod="e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt"
Sep 11 01:15:38.401732 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:38.401711    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd4nd\" (UniqueName: \"kubernetes.io/projected/f75a0b21-b351-4447-8774-e7b34f3fd258-kube-api-access-zd4nd\") pod \"premium-w0-f4de10-kserve-7787487fb8-cswlt\" (UID: \"f75a0b21-b351-4447-8774-e7b34f3fd258\") " pod="e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt"
Sep 11 01:15:38.500445 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:38.500370    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt"
Sep 11 01:15:38.772593 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:38.772564    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt"]
Sep 11 01:15:38.774171 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:15:38.774145    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf75a0b21_b351_4447_8774_e7b34f3fd258.slice/crio-77851c1b50703a876200a4961af839d0c4191ffd2d430baf6abab7061200fbb2 WatchSource:0}: Error finding container 77851c1b50703a876200a4961af839d0c4191ffd2d430baf6abab7061200fbb2: Status 404 returned error can't find the container with id 77851c1b50703a876200a4961af839d0c4191ffd2d430baf6abab7061200fbb2
Sep 11 01:15:38.953883 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:38.953842    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt" event={"ID":"f75a0b21-b351-4447-8774-e7b34f3fd258","Type":"ContainerStarted","Data":"46482d198d8ead12b7d6f94720896cd8788c557d556f5b527287349783435dce"}
Sep 11 01:15:38.953883 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:38.953883    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt" event={"ID":"f75a0b21-b351-4447-8774-e7b34f3fd258","Type":"ContainerStarted","Data":"77851c1b50703a876200a4961af839d0c4191ffd2d430baf6abab7061200fbb2"}
Sep 11 01:15:38.971230 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:38.971182    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt" podStartSLOduration=0.971167668 podStartE2EDuration="971.167668ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:15:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:15:38.969498137 +0000 UTC m=+1054.366690332" watchObservedRunningTime="2026-09-11 01:15:38.971167668 +0000 UTC m=+1054.368359863"
Sep 11 01:15:44.169149 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:44.169111    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:15:48.501027 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:48.500988    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt"
Sep 11 01:15:48.501027 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:48.501037    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt"
Sep 11 01:15:48.513839 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:48.513813    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt"
Sep 11 01:15:49.015035 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:49.015007    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt"
Sep 11 01:15:49.421962 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:49.421933    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:15:55.109636 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:55.109601    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:15:55.247460 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:55.247427    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:15:55.372610 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:15:55.372491    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:16:04.266868 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:04.266757    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:16:05.978469 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:05.978425    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:16:06.888828 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:06.888792    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:16:12.525849 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:12.525821    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:16:12.605062 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:12.605038    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:16:12.815557 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:12.815480    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:16:12.912684 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:12.912649    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:16:13.610453 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:13.610415    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:16:13.707368 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:13.707319    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:16:13.802725 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:13.802687    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:16:15.492426 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:15.492389    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:16:21.388620 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:21.388576    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:16:21.487470 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:21.487438    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:16:22.089079 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:22.089038    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:16:23.875105 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:23.875067    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698"]
Sep 11 01:16:23.879045 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:23.879022    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698"
Sep 11 01:16:23.882510 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:23.882484    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-10a215\"/\"kube-root-ca.crt\""
Sep 11 01:16:23.882644 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:23.882495    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-10a215\"/\"openshift-service-ca.crt\""
Sep 11 01:16:23.882644 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:23.882552    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-10a215\"/\"default-dockercfg-h88qs\""
Sep 11 01:16:23.882644 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:23.882588    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-10a215\"/\"test-model-10a215-kserve-self-signed-certs\""
Sep 11 01:16:23.889115 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:23.889089    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698"]
Sep 11 01:16:23.983437 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:23.983404    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:16:24.014613 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:24.014580    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-972tn\" (UniqueName: \"kubernetes.io/projected/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-kube-api-access-972tn\") pod \"test-model-10a215-kserve-7df754df8f-tn698\" (UID: \"ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f\") " pod="ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698"
Sep 11 01:16:24.014754 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:24.014623    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-tls-certs\") pod \"test-model-10a215-kserve-7df754df8f-tn698\" (UID: \"ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f\") " pod="ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698"
Sep 11 01:16:24.014754 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:24.014697    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-model-cache\") pod \"test-model-10a215-kserve-7df754df8f-tn698\" (UID: \"ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f\") " pod="ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698"
Sep 11 01:16:24.014754 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:24.014743    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-tmp-dir\") pod \"test-model-10a215-kserve-7df754df8f-tn698\" (UID: \"ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f\") " pod="ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698"
Sep 11 01:16:24.014877 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:24.014774    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-dshm\") pod \"test-model-10a215-kserve-7df754df8f-tn698\" (UID: \"ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f\") " pod="ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698"
Sep 11 01:16:24.014877 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:24.014830    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-home\") pod \"test-model-10a215-kserve-7df754df8f-tn698\" (UID: \"ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f\") " pod="ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698"
Sep 11 01:16:24.115852 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:24.115820    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-model-cache\") pod \"test-model-10a215-kserve-7df754df8f-tn698\" (UID: \"ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f\") " pod="ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698"
Sep 11 01:16:24.116045 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:24.115864    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-tmp-dir\") pod \"test-model-10a215-kserve-7df754df8f-tn698\" (UID: \"ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f\") " pod="ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698"
Sep 11 01:16:24.116045 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:24.116002    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-dshm\") pod \"test-model-10a215-kserve-7df754df8f-tn698\" (UID: \"ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f\") " pod="ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698"
Sep 11 01:16:24.116140 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:24.116058    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-home\") pod \"test-model-10a215-kserve-7df754df8f-tn698\" (UID: \"ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f\") " pod="ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698"
Sep 11 01:16:24.116140 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:24.116122    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-972tn\" (UniqueName: \"kubernetes.io/projected/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-kube-api-access-972tn\") pod \"test-model-10a215-kserve-7df754df8f-tn698\" (UID: \"ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f\") " pod="ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698"
Sep 11 01:16:24.116213 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:24.116173    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-tls-certs\") pod \"test-model-10a215-kserve-7df754df8f-tn698\" (UID: \"ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f\") " pod="ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698"
Sep 11 01:16:24.116213 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:24.116193    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-tmp-dir\") pod \"test-model-10a215-kserve-7df754df8f-tn698\" (UID: \"ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f\") " pod="ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698"
Sep 11 01:16:24.116288 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:24.116270    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-model-cache\") pod \"test-model-10a215-kserve-7df754df8f-tn698\" (UID: \"ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f\") " pod="ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698"
Sep 11 01:16:24.116447 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:24.116421    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-home\") pod \"test-model-10a215-kserve-7df754df8f-tn698\" (UID: \"ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f\") " pod="ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698"
Sep 11 01:16:24.118482 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:24.118461    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-dshm\") pod \"test-model-10a215-kserve-7df754df8f-tn698\" (UID: \"ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f\") " pod="ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698"
Sep 11 01:16:24.118719 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:24.118701    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-tls-certs\") pod \"test-model-10a215-kserve-7df754df8f-tn698\" (UID: \"ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f\") " pod="ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698"
Sep 11 01:16:24.124159 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:24.124140    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-972tn\" (UniqueName: \"kubernetes.io/projected/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-kube-api-access-972tn\") pod \"test-model-10a215-kserve-7df754df8f-tn698\" (UID: \"ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f\") " pod="ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698"
Sep 11 01:16:24.189863 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:24.189838    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698"
Sep 11 01:16:24.460487 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:24.460407    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698"]
Sep 11 01:16:24.463845 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:16:24.463819    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podebf1d8a3_5b6c_4684_a95f_d5d0be6c571f.slice/crio-7e36aaf43557494f24689603ee544531176ad9123c6339e6c67599f862c07119 WatchSource:0}: Error finding container 7e36aaf43557494f24689603ee544531176ad9123c6339e6c67599f862c07119: Status 404 returned error can't find the container with id 7e36aaf43557494f24689603ee544531176ad9123c6339e6c67599f862c07119
Sep 11 01:16:25.191521 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:25.191488    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698" event={"ID":"ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f","Type":"ContainerStarted","Data":"3444daec1d551076df1edef44a36da522126a0d8d18310b250ddf078e7e2c7bb"}
Sep 11 01:16:25.191521 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:25.191528    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698" event={"ID":"ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f","Type":"ContainerStarted","Data":"7e36aaf43557494f24689603ee544531176ad9123c6339e6c67599f862c07119"}
Sep 11 01:16:25.208027 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:25.207980    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698" podStartSLOduration=2.207960851 podStartE2EDuration="2.207960851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:16:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:16:25.207292532 +0000 UTC m=+1100.604484707" watchObservedRunningTime="2026-09-11 01:16:25.207960851 +0000 UTC m=+1100.605153026"
Sep 11 01:16:27.884304 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:27.884269    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:16:27.986643 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:27.986606    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:16:28.137853 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:28.137774    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:16:28.479267 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:28.479228    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:16:28.636590 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:28.636554    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:16:34.189993 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:34.189954    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698"
Sep 11 01:16:34.190384 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:34.190009    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698"
Sep 11 01:16:34.202491 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:34.202472    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698"
Sep 11 01:16:34.249614 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:34.249586    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698"
Sep 11 01:16:34.748266 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:34.748233    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:16:34.838632 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:34.838603    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:16:35.976591 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:35.976554    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:16:36.036398 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:36.036367    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:16:36.272807 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:36.272729    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:16:36.463889 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:36.463849    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:16:36.559025 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:36.558948    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:16:36.608487 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:36.608449    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ef3c90aa-59c4f7db7c-5mfmf"]
Sep 11 01:16:36.611826 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:36.611796    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-ef3c90aa-59c4f7db7c-5mfmf"
Sep 11 01:16:36.614117 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:36.614090    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-ef3c90aa\""
Sep 11 01:16:36.614306 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:36.614286    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-ef3c90aa-dockercfg-jdtrr\""
Sep 11 01:16:36.624841 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:36.624802    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ef3c90aa-59c4f7db7c-5mfmf"]
Sep 11 01:16:36.726725 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:36.726682    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkrk2\" (UniqueName: \"kubernetes.io/projected/8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda-kube-api-access-kkrk2\") pod \"payload-processing-e2e-mt-ef3c90aa-59c4f7db7c-5mfmf\" (UID: \"8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda\") " pod="openshift-ingress/payload-processing-e2e-mt-ef3c90aa-59c4f7db7c-5mfmf"
Sep 11 01:16:36.727119 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:36.727094    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda-plugins-config-volume\") pod \"payload-processing-e2e-mt-ef3c90aa-59c4f7db7c-5mfmf\" (UID: \"8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda\") " pod="openshift-ingress/payload-processing-e2e-mt-ef3c90aa-59c4f7db7c-5mfmf"
Sep 11 01:16:36.827799 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:36.827695    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda-plugins-config-volume\") pod \"payload-processing-e2e-mt-ef3c90aa-59c4f7db7c-5mfmf\" (UID: \"8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda\") " pod="openshift-ingress/payload-processing-e2e-mt-ef3c90aa-59c4f7db7c-5mfmf"
Sep 11 01:16:36.827799 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:36.827787    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkrk2\" (UniqueName: \"kubernetes.io/projected/8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda-kube-api-access-kkrk2\") pod \"payload-processing-e2e-mt-ef3c90aa-59c4f7db7c-5mfmf\" (UID: \"8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda\") " pod="openshift-ingress/payload-processing-e2e-mt-ef3c90aa-59c4f7db7c-5mfmf"
Sep 11 01:16:36.828444 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:36.828418    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda-plugins-config-volume\") pod \"payload-processing-e2e-mt-ef3c90aa-59c4f7db7c-5mfmf\" (UID: \"8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda\") " pod="openshift-ingress/payload-processing-e2e-mt-ef3c90aa-59c4f7db7c-5mfmf"
Sep 11 01:16:36.836623 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:36.836598    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkrk2\" (UniqueName: \"kubernetes.io/projected/8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda-kube-api-access-kkrk2\") pod \"payload-processing-e2e-mt-ef3c90aa-59c4f7db7c-5mfmf\" (UID: \"8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda\") " pod="openshift-ingress/payload-processing-e2e-mt-ef3c90aa-59c4f7db7c-5mfmf"
Sep 11 01:16:36.929476 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:36.929442    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-ef3c90aa-59c4f7db7c-5mfmf"
Sep 11 01:16:37.214494 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:37.214459    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ef3c90aa-59c4f7db7c-5mfmf"]
Sep 11 01:16:37.217438 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:16:37.217393    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b3c793d_25f4_4cb5_8ba4_4a11a63b3eda.slice/crio-06dad23433011c4429d2e6cf6517b9c42ff51655038b18a2759aa4de02fdf00b WatchSource:0}: Error finding container 06dad23433011c4429d2e6cf6517b9c42ff51655038b18a2759aa4de02fdf00b: Status 404 returned error can't find the container with id 06dad23433011c4429d2e6cf6517b9c42ff51655038b18a2759aa4de02fdf00b
Sep 11 01:16:37.254624 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:37.254591    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-ef3c90aa-59c4f7db7c-5mfmf" event={"ID":"8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda","Type":"ContainerStarted","Data":"06dad23433011c4429d2e6cf6517b9c42ff51655038b18a2759aa4de02fdf00b"}
Sep 11 01:16:38.261628 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:38.261597    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-ef3c90aa-59c4f7db7c-5mfmf" event={"ID":"8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda","Type":"ContainerStarted","Data":"32e778251ddba08f06fb5614294f488da78b5922151c16ff19dfad855ca016cf"}
Sep 11 01:16:38.262035 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:38.261669    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-ef3c90aa-59c4f7db7c-5mfmf"
Sep 11 01:16:38.281253 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:38.281205    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-ef3c90aa-59c4f7db7c-5mfmf" podStartSLOduration=2.281190777 podStartE2EDuration="2.281190777s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:16:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:16:38.27902383 +0000 UTC m=+1113.676216005" watchObservedRunningTime="2026-09-11 01:16:38.281190777 +0000 UTC m=+1113.678382967"
Sep 11 01:16:49.268498 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:49.268458    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-ef3c90aa-59c4f7db7c-5mfmf"
Sep 11 01:16:59.755513 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:16:59.755477    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:00.445773 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:00.445723    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:00.542511 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:00.542482    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:00.903634 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:00.903532    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:02.604442 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:02.604399    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:02.705245 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:02.705205    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:04.290684 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:04.290646    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs"]
Sep 11 01:17:04.294216 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:04.294194    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs"
Sep 11 01:17:04.297531 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:04.297511    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-987d60\"/\"default-dockercfg-pk9qd\""
Sep 11 01:17:04.297531 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:04.297518    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-987d60\"/\"kube-root-ca.crt\""
Sep 11 01:17:04.297723 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:04.297569    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-987d60\"/\"openshift-service-ca.crt\""
Sep 11 01:17:04.297723 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:04.297514    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-987d60\"/\"explicit-model-987d60-kserve-self-signed-certs\""
Sep 11 01:17:04.305075 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:04.305042    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs"]
Sep 11 01:17:04.313390 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:04.313368    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:04.376923 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:04.376866    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-dshm\") pod \"explicit-model-987d60-kserve-fcc9646bf-5rwfs\" (UID: \"867d4c9a-f2e5-4b99-9c20-8357d98d7cb2\") " pod="ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs"
Sep 11 01:17:04.377116 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:04.376931    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-tls-certs\") pod \"explicit-model-987d60-kserve-fcc9646bf-5rwfs\" (UID: \"867d4c9a-f2e5-4b99-9c20-8357d98d7cb2\") " pod="ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs"
Sep 11 01:17:04.377116 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:04.376961    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg4lp\" (UniqueName: \"kubernetes.io/projected/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-kube-api-access-zg4lp\") pod \"explicit-model-987d60-kserve-fcc9646bf-5rwfs\" (UID: \"867d4c9a-f2e5-4b99-9c20-8357d98d7cb2\") " pod="ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs"
Sep 11 01:17:04.377116 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:04.376998    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-model-cache\") pod \"explicit-model-987d60-kserve-fcc9646bf-5rwfs\" (UID: \"867d4c9a-f2e5-4b99-9c20-8357d98d7cb2\") " pod="ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs"
Sep 11 01:17:04.377116 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:04.377014    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-tmp-dir\") pod \"explicit-model-987d60-kserve-fcc9646bf-5rwfs\" (UID: \"867d4c9a-f2e5-4b99-9c20-8357d98d7cb2\") " pod="ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs"
Sep 11 01:17:04.377317 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:04.377109    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-home\") pod \"explicit-model-987d60-kserve-fcc9646bf-5rwfs\" (UID: \"867d4c9a-f2e5-4b99-9c20-8357d98d7cb2\") " pod="ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs"
Sep 11 01:17:04.478772 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:04.478732    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-dshm\") pod \"explicit-model-987d60-kserve-fcc9646bf-5rwfs\" (UID: \"867d4c9a-f2e5-4b99-9c20-8357d98d7cb2\") " pod="ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs"
Sep 11 01:17:04.478772 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:04.478771    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-tls-certs\") pod \"explicit-model-987d60-kserve-fcc9646bf-5rwfs\" (UID: \"867d4c9a-f2e5-4b99-9c20-8357d98d7cb2\") " pod="ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs"
Sep 11 01:17:04.479090 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:04.478795    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg4lp\" (UniqueName: \"kubernetes.io/projected/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-kube-api-access-zg4lp\") pod \"explicit-model-987d60-kserve-fcc9646bf-5rwfs\" (UID: \"867d4c9a-f2e5-4b99-9c20-8357d98d7cb2\") " pod="ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs"
Sep 11 01:17:04.479090 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:04.478823    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-model-cache\") pod \"explicit-model-987d60-kserve-fcc9646bf-5rwfs\" (UID: \"867d4c9a-f2e5-4b99-9c20-8357d98d7cb2\") " pod="ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs"
Sep 11 01:17:04.479090 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:04.478839    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-tmp-dir\") pod \"explicit-model-987d60-kserve-fcc9646bf-5rwfs\" (UID: \"867d4c9a-f2e5-4b99-9c20-8357d98d7cb2\") " pod="ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs"
Sep 11 01:17:04.479090 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:04.478867    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-home\") pod \"explicit-model-987d60-kserve-fcc9646bf-5rwfs\" (UID: \"867d4c9a-f2e5-4b99-9c20-8357d98d7cb2\") " pod="ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs"
Sep 11 01:17:04.479304 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:04.479282    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-model-cache\") pod \"explicit-model-987d60-kserve-fcc9646bf-5rwfs\" (UID: \"867d4c9a-f2e5-4b99-9c20-8357d98d7cb2\") " pod="ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs"
Sep 11 01:17:04.479379 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:04.479359    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-home\") pod \"explicit-model-987d60-kserve-fcc9646bf-5rwfs\" (UID: \"867d4c9a-f2e5-4b99-9c20-8357d98d7cb2\") " pod="ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs"
Sep 11 01:17:04.479433 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:04.479377    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-tmp-dir\") pod \"explicit-model-987d60-kserve-fcc9646bf-5rwfs\" (UID: \"867d4c9a-f2e5-4b99-9c20-8357d98d7cb2\") " pod="ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs"
Sep 11 01:17:04.481326 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:04.481302    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-dshm\") pod \"explicit-model-987d60-kserve-fcc9646bf-5rwfs\" (UID: \"867d4c9a-f2e5-4b99-9c20-8357d98d7cb2\") " pod="ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs"
Sep 11 01:17:04.481598 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:04.481578    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-tls-certs\") pod \"explicit-model-987d60-kserve-fcc9646bf-5rwfs\" (UID: \"867d4c9a-f2e5-4b99-9c20-8357d98d7cb2\") " pod="ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs"
Sep 11 01:17:04.487232 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:04.487211    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg4lp\" (UniqueName: \"kubernetes.io/projected/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-kube-api-access-zg4lp\") pod \"explicit-model-987d60-kserve-fcc9646bf-5rwfs\" (UID: \"867d4c9a-f2e5-4b99-9c20-8357d98d7cb2\") " pod="ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs"
Sep 11 01:17:04.603499 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:04.603392    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs"
Sep 11 01:17:04.877185 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:04.877157    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs"]
Sep 11 01:17:04.879166 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:17:04.879127    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod867d4c9a_f2e5_4b99_9c20_8357d98d7cb2.slice/crio-bf8fecb0d9012c3359810b6fdaa14d1c4c47187f74ab82c017b588bc855041e5 WatchSource:0}: Error finding container bf8fecb0d9012c3359810b6fdaa14d1c4c47187f74ab82c017b588bc855041e5: Status 404 returned error can't find the container with id bf8fecb0d9012c3359810b6fdaa14d1c4c47187f74ab82c017b588bc855041e5
Sep 11 01:17:05.411374 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:05.411338    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs" event={"ID":"867d4c9a-f2e5-4b99-9c20-8357d98d7cb2","Type":"ContainerStarted","Data":"970e564b78683fdd8b8ef73dd126e5b7ed926775c75c27165e382eeec2ad4b51"}
Sep 11 01:17:05.411374 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:05.411379    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs" event={"ID":"867d4c9a-f2e5-4b99-9c20-8357d98d7cb2","Type":"ContainerStarted","Data":"bf8fecb0d9012c3359810b6fdaa14d1c4c47187f74ab82c017b588bc855041e5"}
Sep 11 01:17:05.428776 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:05.428730    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs" podStartSLOduration=1.4287150880000001 podStartE2EDuration="1.428715088s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:17:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:17:05.426343665 +0000 UTC m=+1140.823535838" watchObservedRunningTime="2026-09-11 01:17:05.428715088 +0000 UTC m=+1140.825907261"
Sep 11 01:17:06.617623 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:06.617583    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:06.686982 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:06.686872    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:06.993732 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:06.993655    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ef3c90aa-6fks9"]
Sep 11 01:17:06.996984 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:06.996967    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ef3c90aa-6fks9"
Sep 11 01:17:06.999377 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:06.999359    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 01:17:06.999448 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:06.999374    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q6gbn\""
Sep 11 01:17:07.005156 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:07.005133    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ef3c90aa-6fks9"]
Sep 11 01:17:07.109474 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:07.109440    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78559a00-993f-4748-b85e-f46da6a68dfe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ef3c90aa-6fks9\" (UID: \"78559a00-993f-4748-b85e-f46da6a68dfe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ef3c90aa-6fks9"
Sep 11 01:17:07.210468 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:07.210425    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78559a00-993f-4748-b85e-f46da6a68dfe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ef3c90aa-6fks9\" (UID: \"78559a00-993f-4748-b85e-f46da6a68dfe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ef3c90aa-6fks9"
Sep 11 01:17:07.211112 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:07.211090    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78559a00-993f-4748-b85e-f46da6a68dfe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ef3c90aa-6fks9\" (UID: \"78559a00-993f-4748-b85e-f46da6a68dfe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ef3c90aa-6fks9"
Sep 11 01:17:07.308964 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:07.308882    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ef3c90aa-6fks9"
Sep 11 01:17:07.580840 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:07.580812    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ef3c90aa-6fks9"]
Sep 11 01:17:07.582547 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:17:07.582515    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78559a00_993f_4748_b85e_f46da6a68dfe.slice/crio-368afdb1a1ab6ebb4b60d918982483fde01109da99da5d5648764c42fe4a9d0e WatchSource:0}: Error finding container 368afdb1a1ab6ebb4b60d918982483fde01109da99da5d5648764c42fe4a9d0e: Status 404 returned error can't find the container with id 368afdb1a1ab6ebb4b60d918982483fde01109da99da5d5648764c42fe4a9d0e
Sep 11 01:17:08.429902 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:08.429865    2574 generic.go:356] "Generic (PLEG): container finished" podID="78559a00-993f-4748-b85e-f46da6a68dfe" containerID="c98534f7ffb57dea64748b958843202510c80fe104379e82d8e2bc2650e3497b" exitCode=0
Sep 11 01:17:08.430370 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:08.429952    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ef3c90aa-6fks9" event={"ID":"78559a00-993f-4748-b85e-f46da6a68dfe","Type":"ContainerDied","Data":"c98534f7ffb57dea64748b958843202510c80fe104379e82d8e2bc2650e3497b"}
Sep 11 01:17:08.430370 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:08.430003    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ef3c90aa-6fks9" event={"ID":"78559a00-993f-4748-b85e-f46da6a68dfe","Type":"ContainerStarted","Data":"368afdb1a1ab6ebb4b60d918982483fde01109da99da5d5648764c42fe4a9d0e"}
Sep 11 01:17:09.571894 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:09.571870    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ef3c90aa-6fks9"
Sep 11 01:17:09.736409 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:09.736314    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78559a00-993f-4748-b85e-f46da6a68dfe-maas-api-service-ca\") pod \"78559a00-993f-4748-b85e-f46da6a68dfe\" (UID: \"78559a00-993f-4748-b85e-f46da6a68dfe\") "
Sep 11 01:17:09.736675 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:09.736652    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78559a00-993f-4748-b85e-f46da6a68dfe-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "78559a00-993f-4748-b85e-f46da6a68dfe" (UID: "78559a00-993f-4748-b85e-f46da6a68dfe"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:17:09.837466 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:09.837424    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78559a00-993f-4748-b85e-f46da6a68dfe-maas-api-service-ca\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:17:10.441042 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:10.441010    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ef3c90aa-6fks9"
Sep 11 01:17:10.441042 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:10.441035    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ef3c90aa-6fks9" event={"ID":"78559a00-993f-4748-b85e-f46da6a68dfe","Type":"ContainerDied","Data":"368afdb1a1ab6ebb4b60d918982483fde01109da99da5d5648764c42fe4a9d0e"}
Sep 11 01:17:10.441260 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:10.441077    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="368afdb1a1ab6ebb4b60d918982483fde01109da99da5d5648764c42fe4a9d0e"
Sep 11 01:17:14.604322 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:14.604285    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs"
Sep 11 01:17:14.604739 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:14.604342    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs"
Sep 11 01:17:14.617343 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:14.617314    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs"
Sep 11 01:17:15.481636 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:15.481607    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs"
Sep 11 01:17:17.600796 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:17.600761    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:17.645701 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:17.645662    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ef3c90aa-59c4f7db7c-5mfmf"]
Sep 11 01:17:17.646014 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:17.645987    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-ef3c90aa-59c4f7db7c-5mfmf" podUID="8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda" containerName="payload-processing" containerID="cri-o://32e778251ddba08f06fb5614294f488da78b5922151c16ff19dfad855ca016cf" gracePeriod=30
Sep 11 01:17:17.698796 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:17.698761    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:17.784715 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:17.784619    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:17.887046 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:17.887015    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:17.895172 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:17.895152    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-ef3c90aa-59c4f7db7c-5mfmf"
Sep 11 01:17:17.910720 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:17.910685    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ef3c90aa-6fks9"]
Sep 11 01:17:17.912736 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:17.912708    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda-plugins-config-volume\") pod \"8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda\" (UID: \"8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda\") "
Sep 11 01:17:17.912892 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:17.912781    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkrk2\" (UniqueName: \"kubernetes.io/projected/8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda-kube-api-access-kkrk2\") pod \"8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda\" (UID: \"8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda\") "
Sep 11 01:17:17.913168 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:17.913143    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda" (UID: "8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:17:17.914774 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:17.914753    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ef3c90aa-6fks9"]
Sep 11 01:17:17.915136 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:17.915117    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda-kube-api-access-kkrk2" (OuterVolumeSpecName: "kube-api-access-kkrk2") pod "8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda" (UID: "8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda"). InnerVolumeSpecName "kube-api-access-kkrk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:17:17.971288 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:17.971254    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:18.014157 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:18.014108    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda-plugins-config-volume\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:17:18.014157 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:18.014153    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkrk2\" (UniqueName: \"kubernetes.io/projected/8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda-kube-api-access-kkrk2\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:17:18.241160 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:18.241124    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:18.349948 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:18.349897    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:18.487543 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:18.487509    2574 generic.go:356] "Generic (PLEG): container finished" podID="8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda" containerID="32e778251ddba08f06fb5614294f488da78b5922151c16ff19dfad855ca016cf" exitCode=0
Sep 11 01:17:18.487727 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:18.487554    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-ef3c90aa-59c4f7db7c-5mfmf"
Sep 11 01:17:18.487727 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:18.487572    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-ef3c90aa-59c4f7db7c-5mfmf" event={"ID":"8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda","Type":"ContainerDied","Data":"32e778251ddba08f06fb5614294f488da78b5922151c16ff19dfad855ca016cf"}
Sep 11 01:17:18.487727 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:18.487616    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-ef3c90aa-59c4f7db7c-5mfmf" event={"ID":"8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda","Type":"ContainerDied","Data":"06dad23433011c4429d2e6cf6517b9c42ff51655038b18a2759aa4de02fdf00b"}
Sep 11 01:17:18.487727 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:18.487637    2574 scope.go:117] "RemoveContainer" containerID="32e778251ddba08f06fb5614294f488da78b5922151c16ff19dfad855ca016cf"
Sep 11 01:17:18.499210 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:18.499181    2574 scope.go:117] "RemoveContainer" containerID="32e778251ddba08f06fb5614294f488da78b5922151c16ff19dfad855ca016cf"
Sep 11 01:17:18.499989 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:17:18.499957    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32e778251ddba08f06fb5614294f488da78b5922151c16ff19dfad855ca016cf\": container with ID starting with 32e778251ddba08f06fb5614294f488da78b5922151c16ff19dfad855ca016cf not found: ID does not exist" containerID="32e778251ddba08f06fb5614294f488da78b5922151c16ff19dfad855ca016cf"
Sep 11 01:17:18.500104 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:18.499999    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32e778251ddba08f06fb5614294f488da78b5922151c16ff19dfad855ca016cf"} err="failed to get container status \"32e778251ddba08f06fb5614294f488da78b5922151c16ff19dfad855ca016cf\": rpc error: code = NotFound desc = could not find container \"32e778251ddba08f06fb5614294f488da78b5922151c16ff19dfad855ca016cf\": container with ID starting with 32e778251ddba08f06fb5614294f488da78b5922151c16ff19dfad855ca016cf not found: ID does not exist"
Sep 11 01:17:18.510257 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:18.510226    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ef3c90aa-59c4f7db7c-5mfmf"]
Sep 11 01:17:18.513484 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:18.513458    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ef3c90aa-59c4f7db7c-5mfmf"]
Sep 11 01:17:19.101331 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:19.101298    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78559a00-993f-4748-b85e-f46da6a68dfe" path="/var/lib/kubelet/pods/78559a00-993f-4748-b85e-f46da6a68dfe/volumes"
Sep 11 01:17:19.101714 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:19.101663    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda" path="/var/lib/kubelet/pods/8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda/volumes"
Sep 11 01:17:24.086722 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:24.086682    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:24.143071 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:24.143031    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:24.348770 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:24.348679    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:24.693518 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:24.693476    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:24.795038 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:24.794991    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:24.916792 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:24.916755    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:30.557179 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:30.557086    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:30.615507 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:30.615467    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:30.698416 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:30.698373    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:30.798030 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:30.798001    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:30.910351 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:30.910321    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:36.465141 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:36.465109    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:36.540839 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:36.540774    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:37.441708 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:37.441667    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:37.676622 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:37.676582    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-987d60-rmtfz"]
Sep 11 01:17:37.677186 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:37.677166    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78559a00-993f-4748-b85e-f46da6a68dfe" containerName="revoke-keys"
Sep 11 01:17:37.677268 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:37.677188    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="78559a00-993f-4748-b85e-f46da6a68dfe" containerName="revoke-keys"
Sep 11 01:17:37.677268 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:37.677209    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda" containerName="payload-processing"
Sep 11 01:17:37.677268 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:37.677217    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda" containerName="payload-processing"
Sep 11 01:17:37.677417 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:37.677322    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b3c793d-25f4-4cb5-8ba4-4a11a63b3eda" containerName="payload-processing"
Sep 11 01:17:37.677417 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:37.677337    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="78559a00-993f-4748-b85e-f46da6a68dfe" containerName="revoke-keys"
Sep 11 01:17:37.680601 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:37.680580    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-987d60-rmtfz"
Sep 11 01:17:37.682569 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:37.682547    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q6gbn\""
Sep 11 01:17:37.682689 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:37.682547    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 01:17:37.687997 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:37.687976    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-987d60-rmtfz"]
Sep 11 01:17:37.787412 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:37.787298    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:37.805544 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:37.805513    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71b15436-fd7e-4ca4-bec7-90a4cc72cbff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-987d60-rmtfz\" (UID: \"71b15436-fd7e-4ca4-bec7-90a4cc72cbff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-987d60-rmtfz"
Sep 11 01:17:37.906844 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:37.906809    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71b15436-fd7e-4ca4-bec7-90a4cc72cbff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-987d60-rmtfz\" (UID: \"71b15436-fd7e-4ca4-bec7-90a4cc72cbff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-987d60-rmtfz"
Sep 11 01:17:37.907457 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:37.907436    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71b15436-fd7e-4ca4-bec7-90a4cc72cbff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-987d60-rmtfz\" (UID: \"71b15436-fd7e-4ca4-bec7-90a4cc72cbff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-987d60-rmtfz"
Sep 11 01:17:37.910774 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:37.910749    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:37.992614 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:37.992587    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-987d60-rmtfz"
Sep 11 01:17:38.010953 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:38.010926    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:38.263802 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:38.263776    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-987d60-rmtfz"]
Sep 11 01:17:38.266185 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:17:38.266160    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71b15436_fd7e_4ca4_bec7_90a4cc72cbff.slice/crio-bc00b5c4e468ed2742c63fa3e01a0bae98756cad552d0a89bb8bfc64e001f18c WatchSource:0}: Error finding container bc00b5c4e468ed2742c63fa3e01a0bae98756cad552d0a89bb8bfc64e001f18c: Status 404 returned error can't find the container with id bc00b5c4e468ed2742c63fa3e01a0bae98756cad552d0a89bb8bfc64e001f18c
Sep 11 01:17:38.597209 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:38.597114    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-987d60-rmtfz" event={"ID":"71b15436-fd7e-4ca4-bec7-90a4cc72cbff","Type":"ContainerStarted","Data":"d5b5212ca7841bbe516845c0281cd1a2352e8322e592a8ff288f112b5930d1dd"}
Sep 11 01:17:38.597209 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:38.597167    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-987d60-rmtfz" event={"ID":"71b15436-fd7e-4ca4-bec7-90a4cc72cbff","Type":"ContainerStarted","Data":"bc00b5c4e468ed2742c63fa3e01a0bae98756cad552d0a89bb8bfc64e001f18c"}
Sep 11 01:17:38.618918 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:38.618873    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-987d60-rmtfz" podStartSLOduration=1.618856707 podStartE2EDuration="1.618856707s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:17:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:17:38.618007317 +0000 UTC m=+1174.015199490" watchObservedRunningTime="2026-09-11 01:17:38.618856707 +0000 UTC m=+1174.016048950"
Sep 11 01:17:39.603756 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:39.603706    2574 generic.go:356] "Generic (PLEG): container finished" podID="71b15436-fd7e-4ca4-bec7-90a4cc72cbff" containerID="d5b5212ca7841bbe516845c0281cd1a2352e8322e592a8ff288f112b5930d1dd" exitCode=0
Sep 11 01:17:39.604229 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:39.603793    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-987d60-rmtfz" event={"ID":"71b15436-fd7e-4ca4-bec7-90a4cc72cbff","Type":"ContainerDied","Data":"d5b5212ca7841bbe516845c0281cd1a2352e8322e592a8ff288f112b5930d1dd"}
Sep 11 01:17:40.747547 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:40.747526    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-987d60-rmtfz"
Sep 11 01:17:40.839852 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:40.839817    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71b15436-fd7e-4ca4-bec7-90a4cc72cbff-maas-api-service-ca\") pod \"71b15436-fd7e-4ca4-bec7-90a4cc72cbff\" (UID: \"71b15436-fd7e-4ca4-bec7-90a4cc72cbff\") "
Sep 11 01:17:40.840182 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:40.840159    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71b15436-fd7e-4ca4-bec7-90a4cc72cbff-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "71b15436-fd7e-4ca4-bec7-90a4cc72cbff" (UID: "71b15436-fd7e-4ca4-bec7-90a4cc72cbff"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:17:40.940841 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:40.940813    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71b15436-fd7e-4ca4-bec7-90a4cc72cbff-maas-api-service-ca\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:17:40.989186 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:40.989124    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:41.085393 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:41.085335    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:41.616530 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:41.616502    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-987d60-rmtfz"
Sep 11 01:17:41.616530 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:41.616521    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-987d60-rmtfz" event={"ID":"71b15436-fd7e-4ca4-bec7-90a4cc72cbff","Type":"ContainerDied","Data":"bc00b5c4e468ed2742c63fa3e01a0bae98756cad552d0a89bb8bfc64e001f18c"}
Sep 11 01:17:41.616765 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:41.616572    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bc00b5c4e468ed2742c63fa3e01a0bae98756cad552d0a89bb8bfc64e001f18c"
Sep 11 01:17:42.252186 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:42.252105    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:42.315087 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:42.315059    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:42.655620 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:42.655544    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:42.764135 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:42.764099    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:42.881755 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:42.881719    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:44.334057 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:44.334010    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:45.492574 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:45.492526    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:47.733999 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:47.733958    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:48.304421 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:48.304383    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:48.340857 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:48.340820    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-987d60-5d8998b46b-gbljd"]
Sep 11 01:17:48.341106 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:48.341075    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-987d60-5d8998b46b-gbljd" podUID="a6ce8479-aa9a-4b41-9d16-5cc95567b933" containerName="payload-pre-processing" containerID="cri-o://388a03f27eb87ea0382d7351446950da05649aac8a6a88ce7af18e28c1726ca0" gracePeriod=30
Sep 11 01:17:48.404939 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:48.404888    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:48.500685 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:48.500650    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-987d60-rmtfz"]
Sep 11 01:17:48.505041 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:48.505015    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:48.505965 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:48.505937    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-987d60-rmtfz"]
Sep 11 01:17:48.563493 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:48.563394    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:48.618016 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:48.617988    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-987d60-5d8998b46b-gbljd"
Sep 11 01:17:48.658812 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:48.658773    2574 generic.go:356] "Generic (PLEG): container finished" podID="a6ce8479-aa9a-4b41-9d16-5cc95567b933" containerID="388a03f27eb87ea0382d7351446950da05649aac8a6a88ce7af18e28c1726ca0" exitCode=0
Sep 11 01:17:48.659012 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:48.658821    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-987d60-5d8998b46b-gbljd"
Sep 11 01:17:48.659012 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:48.658858    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-987d60-5d8998b46b-gbljd" event={"ID":"a6ce8479-aa9a-4b41-9d16-5cc95567b933","Type":"ContainerDied","Data":"388a03f27eb87ea0382d7351446950da05649aac8a6a88ce7af18e28c1726ca0"}
Sep 11 01:17:48.659012 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:48.658931    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-987d60-5d8998b46b-gbljd" event={"ID":"a6ce8479-aa9a-4b41-9d16-5cc95567b933","Type":"ContainerDied","Data":"1829646d973ef5b8b9f2b8128368ab1a030476e0f4b62ad764c91492dbeffcef"}
Sep 11 01:17:48.659012 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:48.658960    2574 scope.go:117] "RemoveContainer" containerID="388a03f27eb87ea0382d7351446950da05649aac8a6a88ce7af18e28c1726ca0"
Sep 11 01:17:48.667796 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:48.667775    2574 scope.go:117] "RemoveContainer" containerID="388a03f27eb87ea0382d7351446950da05649aac8a6a88ce7af18e28c1726ca0"
Sep 11 01:17:48.668099 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:17:48.668078    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"388a03f27eb87ea0382d7351446950da05649aac8a6a88ce7af18e28c1726ca0\": container with ID starting with 388a03f27eb87ea0382d7351446950da05649aac8a6a88ce7af18e28c1726ca0 not found: ID does not exist" containerID="388a03f27eb87ea0382d7351446950da05649aac8a6a88ce7af18e28c1726ca0"
Sep 11 01:17:48.668167 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:48.668108    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"388a03f27eb87ea0382d7351446950da05649aac8a6a88ce7af18e28c1726ca0"} err="failed to get container status \"388a03f27eb87ea0382d7351446950da05649aac8a6a88ce7af18e28c1726ca0\": rpc error: code = NotFound desc = could not find container \"388a03f27eb87ea0382d7351446950da05649aac8a6a88ce7af18e28c1726ca0\": container with ID starting with 388a03f27eb87ea0382d7351446950da05649aac8a6a88ce7af18e28c1726ca0 not found: ID does not exist"
Sep 11 01:17:48.718596 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:48.718569    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6ce8479-aa9a-4b41-9d16-5cc95567b933-plugins-config-volume\") pod \"a6ce8479-aa9a-4b41-9d16-5cc95567b933\" (UID: \"a6ce8479-aa9a-4b41-9d16-5cc95567b933\") "
Sep 11 01:17:48.718703 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:48.718617    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2wbp2\" (UniqueName: \"kubernetes.io/projected/a6ce8479-aa9a-4b41-9d16-5cc95567b933-kube-api-access-2wbp2\") pod \"a6ce8479-aa9a-4b41-9d16-5cc95567b933\" (UID: \"a6ce8479-aa9a-4b41-9d16-5cc95567b933\") "
Sep 11 01:17:48.718932 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:48.718889    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6ce8479-aa9a-4b41-9d16-5cc95567b933-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a6ce8479-aa9a-4b41-9d16-5cc95567b933" (UID: "a6ce8479-aa9a-4b41-9d16-5cc95567b933"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:17:48.719039 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:48.719025    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6ce8479-aa9a-4b41-9d16-5cc95567b933-plugins-config-volume\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:17:48.720779 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:48.720751    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6ce8479-aa9a-4b41-9d16-5cc95567b933-kube-api-access-2wbp2" (OuterVolumeSpecName: "kube-api-access-2wbp2") pod "a6ce8479-aa9a-4b41-9d16-5cc95567b933" (UID: "a6ce8479-aa9a-4b41-9d16-5cc95567b933"). InnerVolumeSpecName "kube-api-access-2wbp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:17:48.819544 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:48.819469    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2wbp2\" (UniqueName: \"kubernetes.io/projected/a6ce8479-aa9a-4b41-9d16-5cc95567b933-kube-api-access-2wbp2\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:17:48.876029 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:48.875993    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:48.980324 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:48.980289    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-987d60-5d8998b46b-gbljd"]
Sep 11 01:17:48.983357 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:48.983335    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-987d60-5d8998b46b-gbljd"]
Sep 11 01:17:49.100972 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:49.100873    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71b15436-fd7e-4ca4-bec7-90a4cc72cbff" path="/var/lib/kubelet/pods/71b15436-fd7e-4ca4-bec7-90a4cc72cbff/volumes"
Sep 11 01:17:49.101271 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:49.101257    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6ce8479-aa9a-4b41-9d16-5cc95567b933" path="/var/lib/kubelet/pods/a6ce8479-aa9a-4b41-9d16-5cc95567b933/volumes"
Sep 11 01:17:53.856046 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:53.856008    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs"]
Sep 11 01:17:53.896625 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:53.896581    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:54.759138 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:54.759105    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:17:54.891985 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:54.891953    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-cd176e-4qt24"]
Sep 11 01:17:54.892375 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:54.892324    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71b15436-fd7e-4ca4-bec7-90a4cc72cbff" containerName="revoke-keys"
Sep 11 01:17:54.892375 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:54.892335    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="71b15436-fd7e-4ca4-bec7-90a4cc72cbff" containerName="revoke-keys"
Sep 11 01:17:54.892375 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:54.892348    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6ce8479-aa9a-4b41-9d16-5cc95567b933" containerName="payload-pre-processing"
Sep 11 01:17:54.892375 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:54.892353    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6ce8479-aa9a-4b41-9d16-5cc95567b933" containerName="payload-pre-processing"
Sep 11 01:17:54.892526 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:54.892409    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6ce8479-aa9a-4b41-9d16-5cc95567b933" containerName="payload-pre-processing"
Sep 11 01:17:54.892526 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:54.892419    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="71b15436-fd7e-4ca4-bec7-90a4cc72cbff" containerName="revoke-keys"
Sep 11 01:17:54.896805 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:54.896787    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-cd176e-4qt24"
Sep 11 01:17:54.900050 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:54.899518    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 01:17:54.900050 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:54.899702    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q6gbn\""
Sep 11 01:17:54.902896 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:54.902873    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-cd176e-4qt24"]
Sep 11 01:17:54.974719 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:54.974687    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1855fb57-1e84-474d-bf62-ecef8cba2b2f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-cd176e-4qt24\" (UID: \"1855fb57-1e84-474d-bf62-ecef8cba2b2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-cd176e-4qt24"
Sep 11 01:17:55.077038 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:55.076923    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1855fb57-1e84-474d-bf62-ecef8cba2b2f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-cd176e-4qt24\" (UID: \"1855fb57-1e84-474d-bf62-ecef8cba2b2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-cd176e-4qt24"
Sep 11 01:17:55.077619 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:55.077599    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1855fb57-1e84-474d-bf62-ecef8cba2b2f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-cd176e-4qt24\" (UID: \"1855fb57-1e84-474d-bf62-ecef8cba2b2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-cd176e-4qt24"
Sep 11 01:17:55.209538 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:55.209501    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-cd176e-4qt24"
Sep 11 01:17:55.481818 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:55.481787    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-cd176e-4qt24"]
Sep 11 01:17:55.484046 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:17:55.484016    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1855fb57_1e84_474d_bf62_ecef8cba2b2f.slice/crio-0ff225fed1d88f45e07fd99c298b369fea3c4dabd5d18542c92f5ca9a9373695 WatchSource:0}: Error finding container 0ff225fed1d88f45e07fd99c298b369fea3c4dabd5d18542c92f5ca9a9373695: Status 404 returned error can't find the container with id 0ff225fed1d88f45e07fd99c298b369fea3c4dabd5d18542c92f5ca9a9373695
Sep 11 01:17:55.705164 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:55.705131    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-cd176e-4qt24" event={"ID":"1855fb57-1e84-474d-bf62-ecef8cba2b2f","Type":"ContainerStarted","Data":"0ff225fed1d88f45e07fd99c298b369fea3c4dabd5d18542c92f5ca9a9373695"}
Sep 11 01:17:56.712616 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:56.712578    2574 generic.go:356] "Generic (PLEG): container finished" podID="1855fb57-1e84-474d-bf62-ecef8cba2b2f" containerID="7f22573b6d59bc474147fdb44e222b292e36c376c4efebc124da8eb9d83f7967" exitCode=0
Sep 11 01:17:56.713050 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:56.712674    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-cd176e-4qt24" event={"ID":"1855fb57-1e84-474d-bf62-ecef8cba2b2f","Type":"ContainerDied","Data":"7f22573b6d59bc474147fdb44e222b292e36c376c4efebc124da8eb9d83f7967"}
Sep 11 01:17:57.844036 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:57.844010    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-cd176e-4qt24"
Sep 11 01:17:57.903749 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:57.903719    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1855fb57-1e84-474d-bf62-ecef8cba2b2f-maas-api-service-ca\") pod \"1855fb57-1e84-474d-bf62-ecef8cba2b2f\" (UID: \"1855fb57-1e84-474d-bf62-ecef8cba2b2f\") "
Sep 11 01:17:57.904097 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:57.904078    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1855fb57-1e84-474d-bf62-ecef8cba2b2f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1855fb57-1e84-474d-bf62-ecef8cba2b2f" (UID: "1855fb57-1e84-474d-bf62-ecef8cba2b2f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:17:58.004826 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:58.004764    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1855fb57-1e84-474d-bf62-ecef8cba2b2f-maas-api-service-ca\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:17:58.724626 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:58.724593    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-cd176e-4qt24"
Sep 11 01:17:58.724802 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:58.724631    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-cd176e-4qt24" event={"ID":"1855fb57-1e84-474d-bf62-ecef8cba2b2f","Type":"ContainerDied","Data":"0ff225fed1d88f45e07fd99c298b369fea3c4dabd5d18542c92f5ca9a9373695"}
Sep 11 01:17:58.724802 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:58.724677    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0ff225fed1d88f45e07fd99c298b369fea3c4dabd5d18542c92f5ca9a9373695"
Sep 11 01:17:59.753590 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:59.753559    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q"]
Sep 11 01:17:59.845041 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:17:59.845006    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:03.367306 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:03.367266    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:05.107637 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:05.107606    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:05.112325 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:05.112299    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cd176e-29818155-9gr52"]
Sep 11 01:18:05.117771 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:05.117739    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cd176e-29818155-9gr52"]
Sep 11 01:18:05.150983 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:05.150954    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-cd176e-79cc7879f4-tg9wg"]
Sep 11 01:18:05.151199 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:05.151170    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd176e-79cc7879f4-tg9wg" podUID="7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7" containerName="payload-pre-processing" containerID="cri-o://82ec67455be51090f4b6d0f48fc21064a26443133d0a643cb47e84f010be2d37" gracePeriod=30
Sep 11 01:18:05.179128 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:05.179070    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:05.278519 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:05.278488    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:05.327699 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:05.327669    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-cd176e-4qt24"]
Sep 11 01:18:05.331961 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:05.331932    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-cd176e-4qt24"]
Sep 11 01:18:05.346329 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:05.346287    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:05.395779 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:05.395756    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd176e-79cc7879f4-tg9wg"
Sep 11 01:18:05.474215 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:05.474193    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7-plugins-config-volume\") pod \"7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7\" (UID: \"7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7\") "
Sep 11 01:18:05.474341 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:05.474255    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xxzk5\" (UniqueName: \"kubernetes.io/projected/7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7-kube-api-access-xxzk5\") pod \"7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7\" (UID: \"7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7\") "
Sep 11 01:18:05.474624 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:05.474599    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7" (UID: "7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:18:05.476576 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:05.476557    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7-kube-api-access-xxzk5" (OuterVolumeSpecName: "kube-api-access-xxzk5") pod "7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7" (UID: "7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7"). InnerVolumeSpecName "kube-api-access-xxzk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:18:05.575725 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:05.575703    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7-plugins-config-volume\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:18:05.575725 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:05.575724    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxzk5\" (UniqueName: \"kubernetes.io/projected/7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7-kube-api-access-xxzk5\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:18:05.703134 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:05.703101    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:05.764147 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:05.764118    2574 generic.go:356] "Generic (PLEG): container finished" podID="7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7" containerID="82ec67455be51090f4b6d0f48fc21064a26443133d0a643cb47e84f010be2d37" exitCode=0
Sep 11 01:18:05.764261 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:05.764164    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd176e-79cc7879f4-tg9wg"
Sep 11 01:18:05.764261 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:05.764161    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd176e-79cc7879f4-tg9wg" event={"ID":"7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7","Type":"ContainerDied","Data":"82ec67455be51090f4b6d0f48fc21064a26443133d0a643cb47e84f010be2d37"}
Sep 11 01:18:05.764349 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:05.764285    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd176e-79cc7879f4-tg9wg" event={"ID":"7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7","Type":"ContainerDied","Data":"591d6ec81fe3d0fbb4de65e56ab80a20c93c6fd732a897ed51f6a80109df39dd"}
Sep 11 01:18:05.764349 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:05.764309    2574 scope.go:117] "RemoveContainer" containerID="82ec67455be51090f4b6d0f48fc21064a26443133d0a643cb47e84f010be2d37"
Sep 11 01:18:05.773280 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:05.773266    2574 scope.go:117] "RemoveContainer" containerID="82ec67455be51090f4b6d0f48fc21064a26443133d0a643cb47e84f010be2d37"
Sep 11 01:18:05.773552 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:18:05.773534    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82ec67455be51090f4b6d0f48fc21064a26443133d0a643cb47e84f010be2d37\": container with ID starting with 82ec67455be51090f4b6d0f48fc21064a26443133d0a643cb47e84f010be2d37 not found: ID does not exist" containerID="82ec67455be51090f4b6d0f48fc21064a26443133d0a643cb47e84f010be2d37"
Sep 11 01:18:05.773606 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:05.773561    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82ec67455be51090f4b6d0f48fc21064a26443133d0a643cb47e84f010be2d37"} err="failed to get container status \"82ec67455be51090f4b6d0f48fc21064a26443133d0a643cb47e84f010be2d37\": rpc error: code = NotFound desc = could not find container \"82ec67455be51090f4b6d0f48fc21064a26443133d0a643cb47e84f010be2d37\": container with ID starting with 82ec67455be51090f4b6d0f48fc21064a26443133d0a643cb47e84f010be2d37 not found: ID does not exist"
Sep 11 01:18:05.785366 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:05.785346    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-cd176e-79cc7879f4-tg9wg"]
Sep 11 01:18:05.789309 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:05.789289    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-cd176e-79cc7879f4-tg9wg"]
Sep 11 01:18:05.982106 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:05.982030    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:06.329457 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:06.329344    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:06.650466 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:06.650361    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:07.015744 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:07.015711    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:07.100830 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:07.100798    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1855fb57-1e84-474d-bf62-ecef8cba2b2f" path="/var/lib/kubelet/pods/1855fb57-1e84-474d-bf62-ecef8cba2b2f/volumes"
Sep 11 01:18:07.101125 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:07.101111    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60473175-dd74-4ba5-8761-2c662884df97" path="/var/lib/kubelet/pods/60473175-dd74-4ba5-8761-2c662884df97/volumes"
Sep 11 01:18:07.101378 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:07.101367    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7" path="/var/lib/kubelet/pods/7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7/volumes"
Sep 11 01:18:08.875954 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:08.875874    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs" podUID="867d4c9a-f2e5-4b99-9c20-8357d98d7cb2" containerName="main" containerID="cri-o://970e564b78683fdd8b8ef73dd126e5b7ed926775c75c27165e382eeec2ad4b51" gracePeriod=45
Sep 11 01:18:09.130743 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:09.130681    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs"
Sep 11 01:18:09.209236 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:09.209206    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-model-cache\") pod \"867d4c9a-f2e5-4b99-9c20-8357d98d7cb2\" (UID: \"867d4c9a-f2e5-4b99-9c20-8357d98d7cb2\") "
Sep 11 01:18:09.209384 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:09.209242    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zg4lp\" (UniqueName: \"kubernetes.io/projected/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-kube-api-access-zg4lp\") pod \"867d4c9a-f2e5-4b99-9c20-8357d98d7cb2\" (UID: \"867d4c9a-f2e5-4b99-9c20-8357d98d7cb2\") "
Sep 11 01:18:09.209384 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:09.209263    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-tmp-dir\") pod \"867d4c9a-f2e5-4b99-9c20-8357d98d7cb2\" (UID: \"867d4c9a-f2e5-4b99-9c20-8357d98d7cb2\") "
Sep 11 01:18:09.209384 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:09.209299    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-dshm\") pod \"867d4c9a-f2e5-4b99-9c20-8357d98d7cb2\" (UID: \"867d4c9a-f2e5-4b99-9c20-8357d98d7cb2\") "
Sep 11 01:18:09.209384 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:09.209322    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-home\") pod \"867d4c9a-f2e5-4b99-9c20-8357d98d7cb2\" (UID: \"867d4c9a-f2e5-4b99-9c20-8357d98d7cb2\") "
Sep 11 01:18:09.209384 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:09.209342    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-tls-certs\") pod \"867d4c9a-f2e5-4b99-9c20-8357d98d7cb2\" (UID: \"867d4c9a-f2e5-4b99-9c20-8357d98d7cb2\") "
Sep 11 01:18:09.209649 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:09.209468    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-model-cache" (OuterVolumeSpecName: "model-cache") pod "867d4c9a-f2e5-4b99-9c20-8357d98d7cb2" (UID: "867d4c9a-f2e5-4b99-9c20-8357d98d7cb2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:18:09.209649 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:09.209555    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-home" (OuterVolumeSpecName: "home") pod "867d4c9a-f2e5-4b99-9c20-8357d98d7cb2" (UID: "867d4c9a-f2e5-4b99-9c20-8357d98d7cb2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:18:09.209649 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:09.209573    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "867d4c9a-f2e5-4b99-9c20-8357d98d7cb2" (UID: "867d4c9a-f2e5-4b99-9c20-8357d98d7cb2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:18:09.209840 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:09.209815    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-model-cache\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:18:09.209895 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:09.209843    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-tmp-dir\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:18:09.209895 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:09.209856    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-home\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:18:09.211572 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:09.211545    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-dshm" (OuterVolumeSpecName: "dshm") pod "867d4c9a-f2e5-4b99-9c20-8357d98d7cb2" (UID: "867d4c9a-f2e5-4b99-9c20-8357d98d7cb2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:18:09.211674 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:09.211635    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-kube-api-access-zg4lp" (OuterVolumeSpecName: "kube-api-access-zg4lp") pod "867d4c9a-f2e5-4b99-9c20-8357d98d7cb2" (UID: "867d4c9a-f2e5-4b99-9c20-8357d98d7cb2"). InnerVolumeSpecName "kube-api-access-zg4lp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:18:09.211715 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:09.211671    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "867d4c9a-f2e5-4b99-9c20-8357d98d7cb2" (UID: "867d4c9a-f2e5-4b99-9c20-8357d98d7cb2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:18:09.311131 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:09.311105    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zg4lp\" (UniqueName: \"kubernetes.io/projected/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-kube-api-access-zg4lp\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:18:09.311131 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:09.311129    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-dshm\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:18:09.311274 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:09.311139    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2-tls-certs\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:18:09.789289 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:09.789246    2574 generic.go:356] "Generic (PLEG): container finished" podID="867d4c9a-f2e5-4b99-9c20-8357d98d7cb2" containerID="970e564b78683fdd8b8ef73dd126e5b7ed926775c75c27165e382eeec2ad4b51" exitCode=0
Sep 11 01:18:09.789289 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:09.789289    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs"
Sep 11 01:18:09.789496 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:09.789332    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs" event={"ID":"867d4c9a-f2e5-4b99-9c20-8357d98d7cb2","Type":"ContainerDied","Data":"970e564b78683fdd8b8ef73dd126e5b7ed926775c75c27165e382eeec2ad4b51"}
Sep 11 01:18:09.789496 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:09.789377    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs" event={"ID":"867d4c9a-f2e5-4b99-9c20-8357d98d7cb2","Type":"ContainerDied","Data":"bf8fecb0d9012c3359810b6fdaa14d1c4c47187f74ab82c017b588bc855041e5"}
Sep 11 01:18:09.789496 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:09.789394    2574 scope.go:117] "RemoveContainer" containerID="970e564b78683fdd8b8ef73dd126e5b7ed926775c75c27165e382eeec2ad4b51"
Sep 11 01:18:09.798585 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:09.798576    2574 scope.go:117] "RemoveContainer" containerID="970e564b78683fdd8b8ef73dd126e5b7ed926775c75c27165e382eeec2ad4b51"
Sep 11 01:18:09.798842 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:18:09.798823    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"970e564b78683fdd8b8ef73dd126e5b7ed926775c75c27165e382eeec2ad4b51\": container with ID starting with 970e564b78683fdd8b8ef73dd126e5b7ed926775c75c27165e382eeec2ad4b51 not found: ID does not exist" containerID="970e564b78683fdd8b8ef73dd126e5b7ed926775c75c27165e382eeec2ad4b51"
Sep 11 01:18:09.798894 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:09.798851    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"970e564b78683fdd8b8ef73dd126e5b7ed926775c75c27165e382eeec2ad4b51"} err="failed to get container status \"970e564b78683fdd8b8ef73dd126e5b7ed926775c75c27165e382eeec2ad4b51\": rpc error: code = NotFound desc = could not find container \"970e564b78683fdd8b8ef73dd126e5b7ed926775c75c27165e382eeec2ad4b51\": container with ID starting with 970e564b78683fdd8b8ef73dd126e5b7ed926775c75c27165e382eeec2ad4b51 not found: ID does not exist"
Sep 11 01:18:09.809879 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:09.809857    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs"]
Sep 11 01:18:09.812396 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:09.812375    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-987d60/explicit-model-987d60-kserve-fcc9646bf-5rwfs"]
Sep 11 01:18:10.110633 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:10.110559    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:10.207554 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:10.207524    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:11.100195 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:11.100163    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="867d4c9a-f2e5-4b99-9c20-8357d98d7cb2" path="/var/lib/kubelet/pods/867d4c9a-f2e5-4b99-9c20-8357d98d7cb2/volumes"
Sep 11 01:18:11.253848 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:11.253810    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:11.353807 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:11.353733    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:11.453180 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:11.453143    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:11.740029 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:11.740001    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:11.823253 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:11.823222    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:11.868316 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:11.868273    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-4f6824-68859fc547-47gtd"]
Sep 11 01:18:11.868656 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:11.868644    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="867d4c9a-f2e5-4b99-9c20-8357d98d7cb2" containerName="main"
Sep 11 01:18:11.868700 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:11.868658    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="867d4c9a-f2e5-4b99-9c20-8357d98d7cb2" containerName="main"
Sep 11 01:18:11.868700 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:11.868676    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7" containerName="payload-pre-processing"
Sep 11 01:18:11.868700 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:11.868681    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7" containerName="payload-pre-processing"
Sep 11 01:18:11.868700 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:11.868690    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1855fb57-1e84-474d-bf62-ecef8cba2b2f" containerName="revoke-keys"
Sep 11 01:18:11.868700 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:11.868696    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1855fb57-1e84-474d-bf62-ecef8cba2b2f" containerName="revoke-keys"
Sep 11 01:18:11.868855 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:11.868748    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7eebe6a6-8aca-4f77-b7b8-2ab5f6ee1bf7" containerName="payload-pre-processing"
Sep 11 01:18:11.868855 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:11.868758    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1855fb57-1e84-474d-bf62-ecef8cba2b2f" containerName="revoke-keys"
Sep 11 01:18:11.868855 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:11.868764    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="867d4c9a-f2e5-4b99-9c20-8357d98d7cb2" containerName="main"
Sep 11 01:18:11.872979 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:11.872961    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f6824-68859fc547-47gtd"
Sep 11 01:18:11.875389 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:11.875372    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-4f6824-dockercfg-j7tl9\""
Sep 11 01:18:11.875469 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:11.875409    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-4f6824\""
Sep 11 01:18:11.881023 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:11.881000    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-4f6824-68859fc547-47gtd"]
Sep 11 01:18:11.917287 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:11.917250    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f6824-647f95d65d-kflvl"]
Sep 11 01:18:11.920746 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:11.920728    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f6824-647f95d65d-kflvl"
Sep 11 01:18:11.928933 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:11.928881    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f6824-647f95d65d-kflvl"]
Sep 11 01:18:11.933632 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:11.933605    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a26faec-cbdf-47c5-a3e0-f52dc9047999-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-4f6824-68859fc547-47gtd\" (UID: \"6a26faec-cbdf-47c5-a3e0-f52dc9047999\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f6824-68859fc547-47gtd"
Sep 11 01:18:11.933742 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:11.933648    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hpds\" (UniqueName: \"kubernetes.io/projected/6a26faec-cbdf-47c5-a3e0-f52dc9047999-kube-api-access-2hpds\") pod \"payload-processing-e2e-shared-b-w2-4f6824-68859fc547-47gtd\" (UID: \"6a26faec-cbdf-47c5-a3e0-f52dc9047999\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f6824-68859fc547-47gtd"
Sep 11 01:18:12.035091 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:12.035006    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a26faec-cbdf-47c5-a3e0-f52dc9047999-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-4f6824-68859fc547-47gtd\" (UID: \"6a26faec-cbdf-47c5-a3e0-f52dc9047999\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f6824-68859fc547-47gtd"
Sep 11 01:18:12.035091 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:12.035044    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hpds\" (UniqueName: \"kubernetes.io/projected/6a26faec-cbdf-47c5-a3e0-f52dc9047999-kube-api-access-2hpds\") pod \"payload-processing-e2e-shared-b-w2-4f6824-68859fc547-47gtd\" (UID: \"6a26faec-cbdf-47c5-a3e0-f52dc9047999\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f6824-68859fc547-47gtd"
Sep 11 01:18:12.035306 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:12.035169    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19cda1a2-f6b5-4856-89a6-12fcb724ea95-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-4f6824-647f95d65d-kflvl\" (UID: \"19cda1a2-f6b5-4856-89a6-12fcb724ea95\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f6824-647f95d65d-kflvl"
Sep 11 01:18:12.035357 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:12.035338    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zlkj\" (UniqueName: \"kubernetes.io/projected/19cda1a2-f6b5-4856-89a6-12fcb724ea95-kube-api-access-2zlkj\") pod \"payload-pre-processing-e2e-shared-b-w2-4f6824-647f95d65d-kflvl\" (UID: \"19cda1a2-f6b5-4856-89a6-12fcb724ea95\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f6824-647f95d65d-kflvl"
Sep 11 01:18:12.035701 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:12.035683    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a26faec-cbdf-47c5-a3e0-f52dc9047999-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-4f6824-68859fc547-47gtd\" (UID: \"6a26faec-cbdf-47c5-a3e0-f52dc9047999\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f6824-68859fc547-47gtd"
Sep 11 01:18:12.043256 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:12.043224    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hpds\" (UniqueName: \"kubernetes.io/projected/6a26faec-cbdf-47c5-a3e0-f52dc9047999-kube-api-access-2hpds\") pod \"payload-processing-e2e-shared-b-w2-4f6824-68859fc547-47gtd\" (UID: \"6a26faec-cbdf-47c5-a3e0-f52dc9047999\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f6824-68859fc547-47gtd"
Sep 11 01:18:12.136591 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:12.136559    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zlkj\" (UniqueName: \"kubernetes.io/projected/19cda1a2-f6b5-4856-89a6-12fcb724ea95-kube-api-access-2zlkj\") pod \"payload-pre-processing-e2e-shared-b-w2-4f6824-647f95d65d-kflvl\" (UID: \"19cda1a2-f6b5-4856-89a6-12fcb724ea95\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f6824-647f95d65d-kflvl"
Sep 11 01:18:12.136738 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:12.136607    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19cda1a2-f6b5-4856-89a6-12fcb724ea95-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-4f6824-647f95d65d-kflvl\" (UID: \"19cda1a2-f6b5-4856-89a6-12fcb724ea95\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f6824-647f95d65d-kflvl"
Sep 11 01:18:12.137309 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:12.137291    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19cda1a2-f6b5-4856-89a6-12fcb724ea95-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-4f6824-647f95d65d-kflvl\" (UID: \"19cda1a2-f6b5-4856-89a6-12fcb724ea95\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f6824-647f95d65d-kflvl"
Sep 11 01:18:12.144421 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:12.144398    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zlkj\" (UniqueName: \"kubernetes.io/projected/19cda1a2-f6b5-4856-89a6-12fcb724ea95-kube-api-access-2zlkj\") pod \"payload-pre-processing-e2e-shared-b-w2-4f6824-647f95d65d-kflvl\" (UID: \"19cda1a2-f6b5-4856-89a6-12fcb724ea95\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f6824-647f95d65d-kflvl"
Sep 11 01:18:12.185029 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:12.184997    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f6824-68859fc547-47gtd"
Sep 11 01:18:12.232785 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:12.232182    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f6824-647f95d65d-kflvl"
Sep 11 01:18:12.402817 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:12.402783    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:12.462498 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:12.462474    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-4f6824-68859fc547-47gtd"]
Sep 11 01:18:12.464633 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:18:12.464606    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a26faec_cbdf_47c5_a3e0_f52dc9047999.slice/crio-54efeb5d7cd4cec6d170d4632510ca31fe5d7b3b647daab72f86c89d71d73692 WatchSource:0}: Error finding container 54efeb5d7cd4cec6d170d4632510ca31fe5d7b3b647daab72f86c89d71d73692: Status 404 returned error can't find the container with id 54efeb5d7cd4cec6d170d4632510ca31fe5d7b3b647daab72f86c89d71d73692
Sep 11 01:18:12.506400 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:12.506363    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:12.517267 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:12.517241    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f6824-647f95d65d-kflvl"]
Sep 11 01:18:12.518822 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:18:12.518799    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19cda1a2_f6b5_4856_89a6_12fcb724ea95.slice/crio-02e270e1959e48c09110eeedcf096da639bd6af271b9a223e384f46d712a1a76 WatchSource:0}: Error finding container 02e270e1959e48c09110eeedcf096da639bd6af271b9a223e384f46d712a1a76: Status 404 returned error can't find the container with id 02e270e1959e48c09110eeedcf096da639bd6af271b9a223e384f46d712a1a76
Sep 11 01:18:12.660621 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:12.660592    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:12.808388 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:12.808299    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f6824-647f95d65d-kflvl" event={"ID":"19cda1a2-f6b5-4856-89a6-12fcb724ea95","Type":"ContainerStarted","Data":"55657a97eefa2aff970a87cccff3aba8292cc569f6e7b6669107daf6c84c7605"}
Sep 11 01:18:12.808388 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:12.808364    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f6824-647f95d65d-kflvl" event={"ID":"19cda1a2-f6b5-4856-89a6-12fcb724ea95","Type":"ContainerStarted","Data":"02e270e1959e48c09110eeedcf096da639bd6af271b9a223e384f46d712a1a76"}
Sep 11 01:18:12.808595 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:12.808396    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f6824-647f95d65d-kflvl"
Sep 11 01:18:12.809896 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:12.809870    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f6824-68859fc547-47gtd" event={"ID":"6a26faec-cbdf-47c5-a3e0-f52dc9047999","Type":"ContainerStarted","Data":"d6a1bd6c7478e632245b69963c181b2287af65cff28db097702d2b43ef4bd620"}
Sep 11 01:18:12.810053 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:12.809932    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f6824-68859fc547-47gtd" event={"ID":"6a26faec-cbdf-47c5-a3e0-f52dc9047999","Type":"ContainerStarted","Data":"54efeb5d7cd4cec6d170d4632510ca31fe5d7b3b647daab72f86c89d71d73692"}
Sep 11 01:18:12.810053 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:12.809963    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f6824-68859fc547-47gtd"
Sep 11 01:18:12.825534 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:12.825493    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f6824-647f95d65d-kflvl" podStartSLOduration=1.825478079 podStartE2EDuration="1.825478079s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:18:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:18:12.823281515 +0000 UTC m=+1208.220473693" watchObservedRunningTime="2026-09-11 01:18:12.825478079 +0000 UTC m=+1208.222670252"
Sep 11 01:18:12.839951 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:12.839893    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f6824-68859fc547-47gtd" podStartSLOduration=1.839879411 podStartE2EDuration="1.839879411s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:18:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:18:12.837819844 +0000 UTC m=+1208.235012018" watchObservedRunningTime="2026-09-11 01:18:12.839879411 +0000 UTC m=+1208.237071583"
Sep 11 01:18:12.899622 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:12.899591    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:14.773268 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:14.773230    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q" podUID="15337767-f54e-4950-904d-01283098bf2b" containerName="main" containerID="cri-o://ca4fa4a178459b9c2deea936b6a3d86cdf6a11f897c4385f4fe893408fd17ce6" gracePeriod=45
Sep 11 01:18:15.033001 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:15.032939    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q"
Sep 11 01:18:15.164519 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:15.164486    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15337767-f54e-4950-904d-01283098bf2b-home\") pod \"15337767-f54e-4950-904d-01283098bf2b\" (UID: \"15337767-f54e-4950-904d-01283098bf2b\") "
Sep 11 01:18:15.164519 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:15.164520    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15337767-f54e-4950-904d-01283098bf2b-dshm\") pod \"15337767-f54e-4950-904d-01283098bf2b\" (UID: \"15337767-f54e-4950-904d-01283098bf2b\") "
Sep 11 01:18:15.164709 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:15.164561    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15337767-f54e-4950-904d-01283098bf2b-tls-certs\") pod \"15337767-f54e-4950-904d-01283098bf2b\" (UID: \"15337767-f54e-4950-904d-01283098bf2b\") "
Sep 11 01:18:15.164709 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:15.164597    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15337767-f54e-4950-904d-01283098bf2b-tmp-dir\") pod \"15337767-f54e-4950-904d-01283098bf2b\" (UID: \"15337767-f54e-4950-904d-01283098bf2b\") "
Sep 11 01:18:15.164709 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:15.164649    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15337767-f54e-4950-904d-01283098bf2b-model-cache\") pod \"15337767-f54e-4950-904d-01283098bf2b\" (UID: \"15337767-f54e-4950-904d-01283098bf2b\") "
Sep 11 01:18:15.164709 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:15.164694    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gt45d\" (UniqueName: \"kubernetes.io/projected/15337767-f54e-4950-904d-01283098bf2b-kube-api-access-gt45d\") pod \"15337767-f54e-4950-904d-01283098bf2b\" (UID: \"15337767-f54e-4950-904d-01283098bf2b\") "
Sep 11 01:18:15.164840 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:15.164707    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15337767-f54e-4950-904d-01283098bf2b-home" (OuterVolumeSpecName: "home") pod "15337767-f54e-4950-904d-01283098bf2b" (UID: "15337767-f54e-4950-904d-01283098bf2b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:18:15.164983 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:15.164953    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15337767-f54e-4950-904d-01283098bf2b-model-cache" (OuterVolumeSpecName: "model-cache") pod "15337767-f54e-4950-904d-01283098bf2b" (UID: "15337767-f54e-4950-904d-01283098bf2b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:18:15.165108 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:15.165038    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15337767-f54e-4950-904d-01283098bf2b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "15337767-f54e-4950-904d-01283098bf2b" (UID: "15337767-f54e-4950-904d-01283098bf2b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:18:15.165108 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:15.165064    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15337767-f54e-4950-904d-01283098bf2b-model-cache\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:18:15.165108 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:15.165084    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15337767-f54e-4950-904d-01283098bf2b-home\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:18:15.166999 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:15.166969    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15337767-f54e-4950-904d-01283098bf2b-kube-api-access-gt45d" (OuterVolumeSpecName: "kube-api-access-gt45d") pod "15337767-f54e-4950-904d-01283098bf2b" (UID: "15337767-f54e-4950-904d-01283098bf2b"). InnerVolumeSpecName "kube-api-access-gt45d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:18:15.167196 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:15.167169    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15337767-f54e-4950-904d-01283098bf2b-dshm" (OuterVolumeSpecName: "dshm") pod "15337767-f54e-4950-904d-01283098bf2b" (UID: "15337767-f54e-4950-904d-01283098bf2b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:18:15.167276 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:15.167199    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15337767-f54e-4950-904d-01283098bf2b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "15337767-f54e-4950-904d-01283098bf2b" (UID: "15337767-f54e-4950-904d-01283098bf2b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:18:15.266503 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:15.266471    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gt45d\" (UniqueName: \"kubernetes.io/projected/15337767-f54e-4950-904d-01283098bf2b-kube-api-access-gt45d\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:18:15.266503 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:15.266498    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15337767-f54e-4950-904d-01283098bf2b-dshm\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:18:15.266503 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:15.266507    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15337767-f54e-4950-904d-01283098bf2b-tls-certs\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:18:15.266719 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:15.266515    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15337767-f54e-4950-904d-01283098bf2b-tmp-dir\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:18:15.828415 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:15.828380    2574 generic.go:356] "Generic (PLEG): container finished" podID="15337767-f54e-4950-904d-01283098bf2b" containerID="ca4fa4a178459b9c2deea936b6a3d86cdf6a11f897c4385f4fe893408fd17ce6" exitCode=0
Sep 11 01:18:15.828820 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:15.828420    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q"
Sep 11 01:18:15.828820 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:15.828464    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q" event={"ID":"15337767-f54e-4950-904d-01283098bf2b","Type":"ContainerDied","Data":"ca4fa4a178459b9c2deea936b6a3d86cdf6a11f897c4385f4fe893408fd17ce6"}
Sep 11 01:18:15.828820 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:15.828503    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q" event={"ID":"15337767-f54e-4950-904d-01283098bf2b","Type":"ContainerDied","Data":"ec1bad9a0208fcdde59d46a322c084501a99c13c113b2528cb717607561b7088"}
Sep 11 01:18:15.828820 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:15.828520    2574 scope.go:117] "RemoveContainer" containerID="ca4fa4a178459b9c2deea936b6a3d86cdf6a11f897c4385f4fe893408fd17ce6"
Sep 11 01:18:15.838611 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:15.838598    2574 scope.go:117] "RemoveContainer" containerID="ca4fa4a178459b9c2deea936b6a3d86cdf6a11f897c4385f4fe893408fd17ce6"
Sep 11 01:18:15.838862 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:18:15.838844    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca4fa4a178459b9c2deea936b6a3d86cdf6a11f897c4385f4fe893408fd17ce6\": container with ID starting with ca4fa4a178459b9c2deea936b6a3d86cdf6a11f897c4385f4fe893408fd17ce6 not found: ID does not exist" containerID="ca4fa4a178459b9c2deea936b6a3d86cdf6a11f897c4385f4fe893408fd17ce6"
Sep 11 01:18:15.838979 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:15.838868    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca4fa4a178459b9c2deea936b6a3d86cdf6a11f897c4385f4fe893408fd17ce6"} err="failed to get container status \"ca4fa4a178459b9c2deea936b6a3d86cdf6a11f897c4385f4fe893408fd17ce6\": rpc error: code = NotFound desc = could not find container \"ca4fa4a178459b9c2deea936b6a3d86cdf6a11f897c4385f4fe893408fd17ce6\": container with ID starting with ca4fa4a178459b9c2deea936b6a3d86cdf6a11f897c4385f4fe893408fd17ce6 not found: ID does not exist"
Sep 11 01:18:15.849551 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:15.849525    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q"]
Sep 11 01:18:15.853623 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:15.853604    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-cd176e/simulator-w4-cd176e-kserve-79d6954b95-29d4q"]
Sep 11 01:18:17.130932 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:17.128044    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15337767-f54e-4950-904d-01283098bf2b" path="/var/lib/kubelet/pods/15337767-f54e-4950-904d-01283098bf2b/volumes"
Sep 11 01:18:23.818620 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:23.818567    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f6824-647f95d65d-kflvl"
Sep 11 01:18:23.819096 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:23.818680    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f6824-68859fc547-47gtd"
Sep 11 01:18:28.682286 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:28.682247    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:30.784014 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:30.783982    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:31.054298 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:31.054206    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:31.518340 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:31.518310    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:33.049508 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:33.049473    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:33.688094 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:33.688064    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:33.786576 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:33.786549    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:33.947234 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:33.947163    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:35.090541 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:35.090506    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:35.895152 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:35.895121    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7962-da7271"]
Sep 11 01:18:35.895498 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:35.895486    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15337767-f54e-4950-904d-01283098bf2b" containerName="main"
Sep 11 01:18:35.895544 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:35.895499    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="15337767-f54e-4950-904d-01283098bf2b" containerName="main"
Sep 11 01:18:35.895586 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:35.895580    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="15337767-f54e-4950-904d-01283098bf2b" containerName="main"
Sep 11 01:18:35.898597 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:35.898581    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7962-da7271"
Sep 11 01:18:35.900748 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:35.900722    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-7gdtv\""
Sep 11 01:18:35.905207 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:35.905181    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7962-da7271"]
Sep 11 01:18:35.950164 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:35.950139    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtkdf\" (UniqueName: \"kubernetes.io/projected/bf8a4d39-aa16-413d-b1f7-3df916341a25-kube-api-access-xtkdf\") pod \"test-curl-7962-da7271\" (UID: \"bf8a4d39-aa16-413d-b1f7-3df916341a25\") " pod="opendatahub/test-curl-7962-da7271"
Sep 11 01:18:36.051440 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:36.051412    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtkdf\" (UniqueName: \"kubernetes.io/projected/bf8a4d39-aa16-413d-b1f7-3df916341a25-kube-api-access-xtkdf\") pod \"test-curl-7962-da7271\" (UID: \"bf8a4d39-aa16-413d-b1f7-3df916341a25\") " pod="opendatahub/test-curl-7962-da7271"
Sep 11 01:18:36.059611 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:36.059587    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtkdf\" (UniqueName: \"kubernetes.io/projected/bf8a4d39-aa16-413d-b1f7-3df916341a25-kube-api-access-xtkdf\") pod \"test-curl-7962-da7271\" (UID: \"bf8a4d39-aa16-413d-b1f7-3df916341a25\") " pod="opendatahub/test-curl-7962-da7271"
Sep 11 01:18:36.063688 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:36.063665    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:36.116030 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:36.115989    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:36.208762 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:36.208727    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7962-da7271"
Sep 11 01:18:36.476605 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:36.476556    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7962-da7271"]
Sep 11 01:18:36.478796 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:18:36.478765    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf8a4d39_aa16_413d_b1f7_3df916341a25.slice/crio-b934894ab6ad9c0a0f1c9798fbc1943e9d925c54be3d5faffc208b69a93a8c6e WatchSource:0}: Error finding container b934894ab6ad9c0a0f1c9798fbc1943e9d925c54be3d5faffc208b69a93a8c6e: Status 404 returned error can't find the container with id b934894ab6ad9c0a0f1c9798fbc1943e9d925c54be3d5faffc208b69a93a8c6e
Sep 11 01:18:36.946594 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:36.946565    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7962-da7271" event={"ID":"bf8a4d39-aa16-413d-b1f7-3df916341a25","Type":"ContainerStarted","Data":"420a7e8e680ec674d62b98db76d521435d14f961b13979021ddeada099de80a6"}
Sep 11 01:18:36.946759 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:36.946602    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7962-da7271" event={"ID":"bf8a4d39-aa16-413d-b1f7-3df916341a25","Type":"ContainerStarted","Data":"b934894ab6ad9c0a0f1c9798fbc1943e9d925c54be3d5faffc208b69a93a8c6e"}
Sep 11 01:18:36.961257 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:36.961211    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7962-da7271" podStartSLOduration=1.961193224 podStartE2EDuration="1.961193224s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:18:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:18:36.958560568 +0000 UTC m=+1232.355752740" watchObservedRunningTime="2026-09-11 01:18:36.961193224 +0000 UTC m=+1232.358385405"
Sep 11 01:18:37.092470 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:37.092442    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:37.549023 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:37.548987    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7962-da7271"]
Sep 11 01:18:37.553408 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:37.553375    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7962-da7271"]
Sep 11 01:18:37.952358 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:18:37.952327    2574 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7962-da7271" container="test-curl-7962-da7271"
Sep 11 01:18:38.957871 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:38.957828    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7962-da7271" podUID="bf8a4d39-aa16-413d-b1f7-3df916341a25" containerName="test-curl-7962-da7271" containerID="cri-o://420a7e8e680ec674d62b98db76d521435d14f961b13979021ddeada099de80a6" gracePeriod=2
Sep 11 01:18:38.959868 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:18:38.959841    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7962-da7271\" is forbidden: User \"system:node:ip-10-0-139-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-235.ec2.internal' and this object" podUID="bf8a4d39-aa16-413d-b1f7-3df916341a25" pod="opendatahub/test-curl-7962-da7271"
Sep 11 01:18:41.137855 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:41.137833    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7962-da7271"
Sep 11 01:18:41.195325 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:41.195291    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xtkdf\" (UniqueName: \"kubernetes.io/projected/bf8a4d39-aa16-413d-b1f7-3df916341a25-kube-api-access-xtkdf\") pod \"bf8a4d39-aa16-413d-b1f7-3df916341a25\" (UID: \"bf8a4d39-aa16-413d-b1f7-3df916341a25\") "
Sep 11 01:18:41.197388 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:41.197365    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf8a4d39-aa16-413d-b1f7-3df916341a25-kube-api-access-xtkdf" (OuterVolumeSpecName: "kube-api-access-xtkdf") pod "bf8a4d39-aa16-413d-b1f7-3df916341a25" (UID: "bf8a4d39-aa16-413d-b1f7-3df916341a25"). InnerVolumeSpecName "kube-api-access-xtkdf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:18:41.295987 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:41.295904    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtkdf\" (UniqueName: \"kubernetes.io/projected/bf8a4d39-aa16-413d-b1f7-3df916341a25-kube-api-access-xtkdf\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:18:41.975380 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:41.975342    2574 generic.go:356] "Generic (PLEG): container finished" podID="bf8a4d39-aa16-413d-b1f7-3df916341a25" containerID="420a7e8e680ec674d62b98db76d521435d14f961b13979021ddeada099de80a6" exitCode=137
Sep 11 01:18:41.975563 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:41.975397    2574 scope.go:117] "RemoveContainer" containerID="420a7e8e680ec674d62b98db76d521435d14f961b13979021ddeada099de80a6"
Sep 11 01:18:41.975563 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:41.975401    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7962-da7271"
Sep 11 01:18:41.984662 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:41.984645    2574 scope.go:117] "RemoveContainer" containerID="420a7e8e680ec674d62b98db76d521435d14f961b13979021ddeada099de80a6"
Sep 11 01:18:41.984949 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:18:41.984899    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"420a7e8e680ec674d62b98db76d521435d14f961b13979021ddeada099de80a6\": container with ID starting with 420a7e8e680ec674d62b98db76d521435d14f961b13979021ddeada099de80a6 not found: ID does not exist" containerID="420a7e8e680ec674d62b98db76d521435d14f961b13979021ddeada099de80a6"
Sep 11 01:18:41.985017 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:41.984962    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"420a7e8e680ec674d62b98db76d521435d14f961b13979021ddeada099de80a6"} err="failed to get container status \"420a7e8e680ec674d62b98db76d521435d14f961b13979021ddeada099de80a6\": rpc error: code = NotFound desc = could not find container \"420a7e8e680ec674d62b98db76d521435d14f961b13979021ddeada099de80a6\": container with ID starting with 420a7e8e680ec674d62b98db76d521435d14f961b13979021ddeada099de80a6 not found: ID does not exist"
Sep 11 01:18:41.985355 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:18:41.985330    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7962-da7271\" is forbidden: User \"system:node:ip-10-0-139-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-235.ec2.internal' and this object" podUID="bf8a4d39-aa16-413d-b1f7-3df916341a25" pod="opendatahub/test-curl-7962-da7271"
Sep 11 01:18:43.100528 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:43.100497    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf8a4d39-aa16-413d-b1f7-3df916341a25" path="/var/lib/kubelet/pods/bf8a4d39-aa16-413d-b1f7-3df916341a25/volumes"
Sep 11 01:18:44.493790 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:44.493757    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:44.806083 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:44.806010    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:45.169972 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:45.169942    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:48.249104 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:48.249066    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:48.349255 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:48.349220    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:49.533373 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:49.533344    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:49.634587 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:49.634547    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:50.088604 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:50.088572    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:50.192205 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:50.192175    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:50.212982 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:50.212955    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6f69f6e6-55f997794-7s9fv"]
Sep 11 01:18:50.213366 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:50.213355    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf8a4d39-aa16-413d-b1f7-3df916341a25" containerName="test-curl-7962-da7271"
Sep 11 01:18:50.213410 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:50.213369    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf8a4d39-aa16-413d-b1f7-3df916341a25" containerName="test-curl-7962-da7271"
Sep 11 01:18:50.213448 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:50.213439    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf8a4d39-aa16-413d-b1f7-3df916341a25" containerName="test-curl-7962-da7271"
Sep 11 01:18:50.217688 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:50.217674    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6f69f6e6-55f997794-7s9fv"
Sep 11 01:18:50.219965 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:50.219945    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-6f69f6e6\""
Sep 11 01:18:50.220101 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:50.220013    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-6f69f6e6-dockercfg-rcb65\""
Sep 11 01:18:50.229067 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:50.229044    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6f69f6e6-55f997794-7s9fv"]
Sep 11 01:18:50.265995 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:50.265894    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:18:50.276127 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:50.276101    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8452ec4c-930e-45e1-824c-7edf9854201c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-6f69f6e6-55f997794-7s9fv\" (UID: \"8452ec4c-930e-45e1-824c-7edf9854201c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6f69f6e6-55f997794-7s9fv"
Sep 11 01:18:50.276259 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:50.276203    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k88bf\" (UniqueName: \"kubernetes.io/projected/8452ec4c-930e-45e1-824c-7edf9854201c-kube-api-access-k88bf\") pod \"payload-pre-processing-e2e-mt-6f69f6e6-55f997794-7s9fv\" (UID: \"8452ec4c-930e-45e1-824c-7edf9854201c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6f69f6e6-55f997794-7s9fv"
Sep 11 01:18:50.377514 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:50.377439    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8452ec4c-930e-45e1-824c-7edf9854201c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-6f69f6e6-55f997794-7s9fv\" (UID: \"8452ec4c-930e-45e1-824c-7edf9854201c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6f69f6e6-55f997794-7s9fv"
Sep 11 01:18:50.377651 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:50.377565    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k88bf\" (UniqueName: \"kubernetes.io/projected/8452ec4c-930e-45e1-824c-7edf9854201c-kube-api-access-k88bf\") pod \"payload-pre-processing-e2e-mt-6f69f6e6-55f997794-7s9fv\" (UID: \"8452ec4c-930e-45e1-824c-7edf9854201c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6f69f6e6-55f997794-7s9fv"
Sep 11 01:18:50.378254 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:50.378237    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8452ec4c-930e-45e1-824c-7edf9854201c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-6f69f6e6-55f997794-7s9fv\" (UID: \"8452ec4c-930e-45e1-824c-7edf9854201c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6f69f6e6-55f997794-7s9fv"
Sep 11 01:18:50.385296 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:50.385270    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k88bf\" (UniqueName: \"kubernetes.io/projected/8452ec4c-930e-45e1-824c-7edf9854201c-kube-api-access-k88bf\") pod \"payload-pre-processing-e2e-mt-6f69f6e6-55f997794-7s9fv\" (UID: \"8452ec4c-930e-45e1-824c-7edf9854201c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6f69f6e6-55f997794-7s9fv"
Sep 11 01:18:50.529033 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:50.529001    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6f69f6e6-55f997794-7s9fv"
Sep 11 01:18:50.802132 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:50.802103    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6f69f6e6-55f997794-7s9fv"]
Sep 11 01:18:50.804579 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:18:50.804549    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8452ec4c_930e_45e1_824c_7edf9854201c.slice/crio-6967ef434d0daa66b0650e936bed7f312779fcf3f1a469a1866411eedbab4341 WatchSource:0}: Error finding container 6967ef434d0daa66b0650e936bed7f312779fcf3f1a469a1866411eedbab4341: Status 404 returned error can't find the container with id 6967ef434d0daa66b0650e936bed7f312779fcf3f1a469a1866411eedbab4341
Sep 11 01:18:51.028335 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:51.028299    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6f69f6e6-55f997794-7s9fv" event={"ID":"8452ec4c-930e-45e1-824c-7edf9854201c","Type":"ContainerStarted","Data":"3d1fe21743850ed265a305f8aef272ae26fa67986cefa82b5fc6930d411949e7"}
Sep 11 01:18:51.028528 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:51.028360    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6f69f6e6-55f997794-7s9fv" event={"ID":"8452ec4c-930e-45e1-824c-7edf9854201c","Type":"ContainerStarted","Data":"6967ef434d0daa66b0650e936bed7f312779fcf3f1a469a1866411eedbab4341"}
Sep 11 01:18:51.028528 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:51.028434    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-6f69f6e6-55f997794-7s9fv"
Sep 11 01:18:51.045594 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:18:51.045551    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-6f69f6e6-55f997794-7s9fv" podStartSLOduration=1.045536251 podStartE2EDuration="1.045536251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:18:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:18:51.043795834 +0000 UTC m=+1246.440988045" watchObservedRunningTime="2026-09-11 01:18:51.045536251 +0000 UTC m=+1246.442728435"
Sep 11 01:19:02.036460 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:02.036381    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-6f69f6e6-55f997794-7s9fv"
Sep 11 01:19:05.269039 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:05.269006    2574 scope.go:117] "RemoveContainer" containerID="70d1cc586ab9affe3a53d162be33b75356a6da3dc0697f53fdda080a1a726d4b"
Sep 11 01:19:13.245113 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:13.245077    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:19:22.094007 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:22.093973    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:19:39.675727 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:39.675687    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8"]
Sep 11 01:19:39.679348 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:39.679330    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8"
Sep 11 01:19:39.681442 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:39.681422    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-f4de10\"/\"distinct-2-w0-f4de10-kserve-self-signed-certs\""
Sep 11 01:19:39.688844 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:39.688818    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8"]
Sep 11 01:19:39.777709 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:39.777676    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:19:39.810009 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:39.809977    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7159c6eb-0eac-492d-a5a0-720156e16245-dshm\") pod \"distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8\" (UID: \"7159c6eb-0eac-492d-a5a0-720156e16245\") " pod="e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8"
Sep 11 01:19:39.810160 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:39.810040    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7159c6eb-0eac-492d-a5a0-720156e16245-model-cache\") pod \"distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8\" (UID: \"7159c6eb-0eac-492d-a5a0-720156e16245\") " pod="e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8"
Sep 11 01:19:39.810160 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:39.810110    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7h9g\" (UniqueName: \"kubernetes.io/projected/7159c6eb-0eac-492d-a5a0-720156e16245-kube-api-access-x7h9g\") pod \"distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8\" (UID: \"7159c6eb-0eac-492d-a5a0-720156e16245\") " pod="e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8"
Sep 11 01:19:39.810160 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:39.810146    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7159c6eb-0eac-492d-a5a0-720156e16245-home\") pod \"distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8\" (UID: \"7159c6eb-0eac-492d-a5a0-720156e16245\") " pod="e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8"
Sep 11 01:19:39.810273 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:39.810168    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7159c6eb-0eac-492d-a5a0-720156e16245-tls-certs\") pod \"distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8\" (UID: \"7159c6eb-0eac-492d-a5a0-720156e16245\") " pod="e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8"
Sep 11 01:19:39.810273 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:39.810211    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7159c6eb-0eac-492d-a5a0-720156e16245-tmp-dir\") pod \"distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8\" (UID: \"7159c6eb-0eac-492d-a5a0-720156e16245\") " pod="e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8"
Sep 11 01:19:39.911276 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:39.911236    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7159c6eb-0eac-492d-a5a0-720156e16245-dshm\") pod \"distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8\" (UID: \"7159c6eb-0eac-492d-a5a0-720156e16245\") " pod="e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8"
Sep 11 01:19:39.911481 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:39.911295    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7159c6eb-0eac-492d-a5a0-720156e16245-model-cache\") pod \"distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8\" (UID: \"7159c6eb-0eac-492d-a5a0-720156e16245\") " pod="e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8"
Sep 11 01:19:39.911481 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:39.911322    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7h9g\" (UniqueName: \"kubernetes.io/projected/7159c6eb-0eac-492d-a5a0-720156e16245-kube-api-access-x7h9g\") pod \"distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8\" (UID: \"7159c6eb-0eac-492d-a5a0-720156e16245\") " pod="e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8"
Sep 11 01:19:39.911481 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:39.911338    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7159c6eb-0eac-492d-a5a0-720156e16245-home\") pod \"distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8\" (UID: \"7159c6eb-0eac-492d-a5a0-720156e16245\") " pod="e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8"
Sep 11 01:19:39.911481 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:39.911353    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7159c6eb-0eac-492d-a5a0-720156e16245-tls-certs\") pod \"distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8\" (UID: \"7159c6eb-0eac-492d-a5a0-720156e16245\") " pod="e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8"
Sep 11 01:19:39.911481 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:39.911380    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7159c6eb-0eac-492d-a5a0-720156e16245-tmp-dir\") pod \"distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8\" (UID: \"7159c6eb-0eac-492d-a5a0-720156e16245\") " pod="e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8"
Sep 11 01:19:39.911809 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:39.911774    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7159c6eb-0eac-492d-a5a0-720156e16245-model-cache\") pod \"distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8\" (UID: \"7159c6eb-0eac-492d-a5a0-720156e16245\") " pod="e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8"
Sep 11 01:19:39.911809 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:39.911799    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7159c6eb-0eac-492d-a5a0-720156e16245-home\") pod \"distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8\" (UID: \"7159c6eb-0eac-492d-a5a0-720156e16245\") " pod="e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8"
Sep 11 01:19:39.912025 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:39.911830    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7159c6eb-0eac-492d-a5a0-720156e16245-tmp-dir\") pod \"distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8\" (UID: \"7159c6eb-0eac-492d-a5a0-720156e16245\") " pod="e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8"
Sep 11 01:19:39.913770 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:39.913743    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7159c6eb-0eac-492d-a5a0-720156e16245-dshm\") pod \"distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8\" (UID: \"7159c6eb-0eac-492d-a5a0-720156e16245\") " pod="e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8"
Sep 11 01:19:39.913947 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:39.913930    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7159c6eb-0eac-492d-a5a0-720156e16245-tls-certs\") pod \"distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8\" (UID: \"7159c6eb-0eac-492d-a5a0-720156e16245\") " pod="e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8"
Sep 11 01:19:39.921505 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:39.921478    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7h9g\" (UniqueName: \"kubernetes.io/projected/7159c6eb-0eac-492d-a5a0-720156e16245-kube-api-access-x7h9g\") pod \"distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8\" (UID: \"7159c6eb-0eac-492d-a5a0-720156e16245\") " pod="e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8"
Sep 11 01:19:39.988738 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:39.988649    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8"
Sep 11 01:19:40.260573 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:40.260549    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8"]
Sep 11 01:19:40.262493 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:19:40.262461    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7159c6eb_0eac_492d_a5a0_720156e16245.slice/crio-cf02cfbb98d9ee56484d43d36b6b9d240d2d7bc083c8945f7dd3f914a230123f WatchSource:0}: Error finding container cf02cfbb98d9ee56484d43d36b6b9d240d2d7bc083c8945f7dd3f914a230123f: Status 404 returned error can't find the container with id cf02cfbb98d9ee56484d43d36b6b9d240d2d7bc083c8945f7dd3f914a230123f
Sep 11 01:19:40.292447 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:40.292416    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8" event={"ID":"7159c6eb-0eac-492d-a5a0-720156e16245","Type":"ContainerStarted","Data":"cf02cfbb98d9ee56484d43d36b6b9d240d2d7bc083c8945f7dd3f914a230123f"}
Sep 11 01:19:41.299736 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:41.299700    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8" event={"ID":"7159c6eb-0eac-492d-a5a0-720156e16245","Type":"ContainerStarted","Data":"e0e9e0bcc9764ae655a745924cbff741c6ede6873ac7014e1ef9481d6607d554"}
Sep 11 01:19:41.316289 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:41.316240    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8" podStartSLOduration=2.3162225960000002 podStartE2EDuration="2.316222596s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:19:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:19:41.314490699 +0000 UTC m=+1296.711682876" watchObservedRunningTime="2026-09-11 01:19:41.316222596 +0000 UTC m=+1296.713414770"
Sep 11 01:19:49.988976 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:49.988937    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8"
Sep 11 01:19:49.989551 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:49.989001    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8"
Sep 11 01:19:50.001591 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:50.001566    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8"
Sep 11 01:19:50.358374 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:50.358299    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8"
Sep 11 01:19:50.481189 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:50.481159    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:19:58.012033 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:19:58.011989    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:20:05.304114 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:05.304082    2574 scope.go:117] "RemoveContainer" containerID="9f25c1aa6cc2c9f89ca7080b5a4966a38ad9dd21836bd0ce36b174cc7c161226"
Sep 11 01:20:12.079858 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:12.079814    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt"]
Sep 11 01:20:12.085454 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:12.085425    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt"
Sep 11 01:20:12.088226 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:12.088198    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-f4de10\"/\"embedding-w0-f4de10-kserve-self-signed-certs\""
Sep 11 01:20:12.092617 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:12.092592    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt"]
Sep 11 01:20:12.096816 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:12.096787    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-model-cache\") pod \"embedding-w0-f4de10-kserve-d4f94cb44-kztkt\" (UID: \"3e8b3e1d-7599-4c9f-bea1-e921bbb11d91\") " pod="e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt"
Sep 11 01:20:12.096971 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:12.096886    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-tls-certs\") pod \"embedding-w0-f4de10-kserve-d4f94cb44-kztkt\" (UID: \"3e8b3e1d-7599-4c9f-bea1-e921bbb11d91\") " pod="e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt"
Sep 11 01:20:12.096971 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:12.096942    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-dshm\") pod \"embedding-w0-f4de10-kserve-d4f94cb44-kztkt\" (UID: \"3e8b3e1d-7599-4c9f-bea1-e921bbb11d91\") " pod="e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt"
Sep 11 01:20:12.097096 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:12.097003    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-home\") pod \"embedding-w0-f4de10-kserve-d4f94cb44-kztkt\" (UID: \"3e8b3e1d-7599-4c9f-bea1-e921bbb11d91\") " pod="e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt"
Sep 11 01:20:12.097192 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:12.097163    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-tmp-dir\") pod \"embedding-w0-f4de10-kserve-d4f94cb44-kztkt\" (UID: \"3e8b3e1d-7599-4c9f-bea1-e921bbb11d91\") " pod="e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt"
Sep 11 01:20:12.097246 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:12.097230    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dvhk\" (UniqueName: \"kubernetes.io/projected/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-kube-api-access-5dvhk\") pod \"embedding-w0-f4de10-kserve-d4f94cb44-kztkt\" (UID: \"3e8b3e1d-7599-4c9f-bea1-e921bbb11d91\") " pod="e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt"
Sep 11 01:20:12.122886 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:12.122853    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:20:12.197992 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:12.197953    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-tmp-dir\") pod \"embedding-w0-f4de10-kserve-d4f94cb44-kztkt\" (UID: \"3e8b3e1d-7599-4c9f-bea1-e921bbb11d91\") " pod="e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt"
Sep 11 01:20:12.197992 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:12.197996    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dvhk\" (UniqueName: \"kubernetes.io/projected/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-kube-api-access-5dvhk\") pod \"embedding-w0-f4de10-kserve-d4f94cb44-kztkt\" (UID: \"3e8b3e1d-7599-4c9f-bea1-e921bbb11d91\") " pod="e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt"
Sep 11 01:20:12.198249 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:12.198038    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-model-cache\") pod \"embedding-w0-f4de10-kserve-d4f94cb44-kztkt\" (UID: \"3e8b3e1d-7599-4c9f-bea1-e921bbb11d91\") " pod="e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt"
Sep 11 01:20:12.198249 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:12.198066    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-tls-certs\") pod \"embedding-w0-f4de10-kserve-d4f94cb44-kztkt\" (UID: \"3e8b3e1d-7599-4c9f-bea1-e921bbb11d91\") " pod="e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt"
Sep 11 01:20:12.198249 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:12.198084    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-dshm\") pod \"embedding-w0-f4de10-kserve-d4f94cb44-kztkt\" (UID: \"3e8b3e1d-7599-4c9f-bea1-e921bbb11d91\") " pod="e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt"
Sep 11 01:20:12.198249 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:12.198109    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-home\") pod \"embedding-w0-f4de10-kserve-d4f94cb44-kztkt\" (UID: \"3e8b3e1d-7599-4c9f-bea1-e921bbb11d91\") " pod="e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt"
Sep 11 01:20:12.198452 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:12.198371    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-tmp-dir\") pod \"embedding-w0-f4de10-kserve-d4f94cb44-kztkt\" (UID: \"3e8b3e1d-7599-4c9f-bea1-e921bbb11d91\") " pod="e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt"
Sep 11 01:20:12.198452 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:12.198397    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-model-cache\") pod \"embedding-w0-f4de10-kserve-d4f94cb44-kztkt\" (UID: \"3e8b3e1d-7599-4c9f-bea1-e921bbb11d91\") " pod="e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt"
Sep 11 01:20:12.198575 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:12.198488    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-home\") pod \"embedding-w0-f4de10-kserve-d4f94cb44-kztkt\" (UID: \"3e8b3e1d-7599-4c9f-bea1-e921bbb11d91\") " pod="e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt"
Sep 11 01:20:12.200492 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:12.200469    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-dshm\") pod \"embedding-w0-f4de10-kserve-d4f94cb44-kztkt\" (UID: \"3e8b3e1d-7599-4c9f-bea1-e921bbb11d91\") " pod="e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt"
Sep 11 01:20:12.200748 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:12.200729    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-tls-certs\") pod \"embedding-w0-f4de10-kserve-d4f94cb44-kztkt\" (UID: \"3e8b3e1d-7599-4c9f-bea1-e921bbb11d91\") " pod="e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt"
Sep 11 01:20:12.207640 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:12.207615    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dvhk\" (UniqueName: \"kubernetes.io/projected/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-kube-api-access-5dvhk\") pod \"embedding-w0-f4de10-kserve-d4f94cb44-kztkt\" (UID: \"3e8b3e1d-7599-4c9f-bea1-e921bbb11d91\") " pod="e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt"
Sep 11 01:20:12.398043 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:12.397947    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt"
Sep 11 01:20:12.673803 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:12.673773    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt"]
Sep 11 01:20:12.675859 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:20:12.675834    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3e8b3e1d_7599_4c9f_bea1_e921bbb11d91.slice/crio-85b16bc62f4d5756e85ba71ac371db20ffc5e91a8f5979ee15f6251dbe1fecea WatchSource:0}: Error finding container 85b16bc62f4d5756e85ba71ac371db20ffc5e91a8f5979ee15f6251dbe1fecea: Status 404 returned error can't find the container with id 85b16bc62f4d5756e85ba71ac371db20ffc5e91a8f5979ee15f6251dbe1fecea
Sep 11 01:20:13.481043 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:13.481004    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt" event={"ID":"3e8b3e1d-7599-4c9f-bea1-e921bbb11d91","Type":"ContainerStarted","Data":"211a95460b552196bd87a31d0ed6801ddea4601b1f9aa5a697f2f2d0d2bc60d1"}
Sep 11 01:20:13.481043 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:13.481046    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt" event={"ID":"3e8b3e1d-7599-4c9f-bea1-e921bbb11d91","Type":"ContainerStarted","Data":"85b16bc62f4d5756e85ba71ac371db20ffc5e91a8f5979ee15f6251dbe1fecea"}
Sep 11 01:20:13.499156 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:13.499102    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt" podStartSLOduration=1.499086155 podStartE2EDuration="1.499086155s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:20:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:20:13.497467518 +0000 UTC m=+1328.894659716" watchObservedRunningTime="2026-09-11 01:20:13.499086155 +0000 UTC m=+1328.896278360"
Sep 11 01:20:16.478727 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:16.478693    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698"]
Sep 11 01:20:16.502812 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:16.502781    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:20:16.604840 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:16.604810    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e98d1e-f4z7r"]
Sep 11 01:20:16.608534 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:16.608518    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e98d1e-f4z7r"
Sep 11 01:20:16.610782 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:16.610758    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 01:20:16.610782 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:16.610776    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q6gbn\""
Sep 11 01:20:16.616135 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:16.616111    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e98d1e-f4z7r"]
Sep 11 01:20:16.632651 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:16.632628    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7dea1f0a-45c3-4a3d-b84c-ae450612c7de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e98d1e-f4z7r\" (UID: \"7dea1f0a-45c3-4a3d-b84c-ae450612c7de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e98d1e-f4z7r"
Sep 11 01:20:16.733737 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:16.733656    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7dea1f0a-45c3-4a3d-b84c-ae450612c7de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e98d1e-f4z7r\" (UID: \"7dea1f0a-45c3-4a3d-b84c-ae450612c7de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e98d1e-f4z7r"
Sep 11 01:20:16.734281 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:16.734261    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7dea1f0a-45c3-4a3d-b84c-ae450612c7de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e98d1e-f4z7r\" (UID: \"7dea1f0a-45c3-4a3d-b84c-ae450612c7de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e98d1e-f4z7r"
Sep 11 01:20:16.736341 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:16.736316    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:20:16.920459 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:16.920421    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e98d1e-f4z7r"
Sep 11 01:20:17.208450 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:20:17.208387    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dea1f0a_45c3_4a3d_b84c_ae450612c7de.slice/crio-ee09de5ba4ac6fe75fb55723f730a18a8828ad98fdfd0cc963f43409a1c44239 WatchSource:0}: Error finding container ee09de5ba4ac6fe75fb55723f730a18a8828ad98fdfd0cc963f43409a1c44239: Status 404 returned error can't find the container with id ee09de5ba4ac6fe75fb55723f730a18a8828ad98fdfd0cc963f43409a1c44239
Sep 11 01:20:17.210866 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:17.210836    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e98d1e-f4z7r"]
Sep 11 01:20:17.502419 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:17.502387    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e98d1e-f4z7r" event={"ID":"7dea1f0a-45c3-4a3d-b84c-ae450612c7de","Type":"ContainerStarted","Data":"0590f870511ca435649b94cc54b4012b7d53af3c9c7387356d79a0520ad8fdaa"}
Sep 11 01:20:17.502735 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:17.502447    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e98d1e-f4z7r" event={"ID":"7dea1f0a-45c3-4a3d-b84c-ae450612c7de","Type":"ContainerStarted","Data":"ee09de5ba4ac6fe75fb55723f730a18a8828ad98fdfd0cc963f43409a1c44239"}
Sep 11 01:20:17.520383 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:17.520334    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e98d1e-f4z7r" podStartSLOduration=1.5203166449999999 podStartE2EDuration="1.520316645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:20:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:20:17.517870998 +0000 UTC m=+1332.915063173" watchObservedRunningTime="2026-09-11 01:20:17.520316645 +0000 UTC m=+1332.917508820"
Sep 11 01:20:18.510117 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:18.510081    2574 generic.go:356] "Generic (PLEG): container finished" podID="7dea1f0a-45c3-4a3d-b84c-ae450612c7de" containerID="0590f870511ca435649b94cc54b4012b7d53af3c9c7387356d79a0520ad8fdaa" exitCode=0
Sep 11 01:20:18.510585 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:18.510128    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e98d1e-f4z7r" event={"ID":"7dea1f0a-45c3-4a3d-b84c-ae450612c7de","Type":"ContainerDied","Data":"0590f870511ca435649b94cc54b4012b7d53af3c9c7387356d79a0520ad8fdaa"}
Sep 11 01:20:18.783217 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:18.783142    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:20:19.657579 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:19.657557    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e98d1e-f4z7r"
Sep 11 01:20:19.658721 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:19.658694    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7dea1f0a-45c3-4a3d-b84c-ae450612c7de-maas-api-service-ca\") pod \"7dea1f0a-45c3-4a3d-b84c-ae450612c7de\" (UID: \"7dea1f0a-45c3-4a3d-b84c-ae450612c7de\") "
Sep 11 01:20:19.659042 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:19.659023    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7dea1f0a-45c3-4a3d-b84c-ae450612c7de-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7dea1f0a-45c3-4a3d-b84c-ae450612c7de" (UID: "7dea1f0a-45c3-4a3d-b84c-ae450612c7de"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:20:19.759850 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:19.759821    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7dea1f0a-45c3-4a3d-b84c-ae450612c7de-maas-api-service-ca\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:20:20.528268 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:20.528240    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e98d1e-f4z7r"
Sep 11 01:20:20.528268 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:20.528255    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e98d1e-f4z7r" event={"ID":"7dea1f0a-45c3-4a3d-b84c-ae450612c7de","Type":"ContainerDied","Data":"ee09de5ba4ac6fe75fb55723f730a18a8828ad98fdfd0cc963f43409a1c44239"}
Sep 11 01:20:20.528511 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:20.528323    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ee09de5ba4ac6fe75fb55723f730a18a8828ad98fdfd0cc963f43409a1c44239"
Sep 11 01:20:22.398760 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:22.398729    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt"
Sep 11 01:20:22.399323 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:22.398794    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt"
Sep 11 01:20:22.411828 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:22.411803    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt"
Sep 11 01:20:22.550066 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:22.550039    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt"
Sep 11 01:20:24.810713 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:24.810670    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:20:27.279440 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:27.279340    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:20:27.305128 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:27.305097    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e98d1e-65864c5fb5-ggszd"]
Sep 11 01:20:27.305404 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:27.305359    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-e98d1e-65864c5fb5-ggszd" podUID="45a3629e-56a1-41b3-892f-8b2d7269e996" containerName="payload-processing" containerID="cri-o://fb01b84ebb7eaeb9c459aca71eb842c7c6707ed4e13f86c55e4aed3cd0d6ec0c" gracePeriod=30
Sep 11 01:20:27.312546 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:27.312517    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e98d1e-98bcf4-brhgp"]
Sep 11 01:20:27.312818 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:27.312771    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e98d1e-98bcf4-brhgp" podUID="28c374e4-7a51-4884-aa7f-8080b6d30473" containerName="payload-pre-processing" containerID="cri-o://78c81e3c5ff97d6813c3516509b7aac909071720a22d48068957f5a06d32f2bc" gracePeriod=30
Sep 11 01:20:27.377439 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:27.377398    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:20:27.477465 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:27.477431    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:20:27.488813 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:27.488667    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e98d1e-f4z7r"]
Sep 11 01:20:27.491297 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:27.491275    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e98d1e-f4z7r"]
Sep 11 01:20:27.547965 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:27.547870    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:20:27.569169 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:27.569139    2574 generic.go:356] "Generic (PLEG): container finished" podID="28c374e4-7a51-4884-aa7f-8080b6d30473" containerID="78c81e3c5ff97d6813c3516509b7aac909071720a22d48068957f5a06d32f2bc" exitCode=0
Sep 11 01:20:27.569303 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:27.569212    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e98d1e-98bcf4-brhgp" event={"ID":"28c374e4-7a51-4884-aa7f-8080b6d30473","Type":"ContainerDied","Data":"78c81e3c5ff97d6813c3516509b7aac909071720a22d48068957f5a06d32f2bc"}
Sep 11 01:20:27.570721 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:27.570700    2574 generic.go:356] "Generic (PLEG): container finished" podID="45a3629e-56a1-41b3-892f-8b2d7269e996" containerID="fb01b84ebb7eaeb9c459aca71eb842c7c6707ed4e13f86c55e4aed3cd0d6ec0c" exitCode=0
Sep 11 01:20:27.570840 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:27.570774    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e98d1e-65864c5fb5-ggszd" event={"ID":"45a3629e-56a1-41b3-892f-8b2d7269e996","Type":"ContainerDied","Data":"fb01b84ebb7eaeb9c459aca71eb842c7c6707ed4e13f86c55e4aed3cd0d6ec0c"}
Sep 11 01:20:27.595685 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:27.595667    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e98d1e-65864c5fb5-ggszd"
Sep 11 01:20:27.598895 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:27.598877    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e98d1e-98bcf4-brhgp"
Sep 11 01:20:27.634937 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:27.634898    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/45a3629e-56a1-41b3-892f-8b2d7269e996-plugins-config-volume\") pod \"45a3629e-56a1-41b3-892f-8b2d7269e996\" (UID: \"45a3629e-56a1-41b3-892f-8b2d7269e996\") "
Sep 11 01:20:27.635077 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:27.634948    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfg24\" (UniqueName: \"kubernetes.io/projected/45a3629e-56a1-41b3-892f-8b2d7269e996-kube-api-access-gfg24\") pod \"45a3629e-56a1-41b3-892f-8b2d7269e996\" (UID: \"45a3629e-56a1-41b3-892f-8b2d7269e996\") "
Sep 11 01:20:27.635129 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:27.635072    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlb7q\" (UniqueName: \"kubernetes.io/projected/28c374e4-7a51-4884-aa7f-8080b6d30473-kube-api-access-tlb7q\") pod \"28c374e4-7a51-4884-aa7f-8080b6d30473\" (UID: \"28c374e4-7a51-4884-aa7f-8080b6d30473\") "
Sep 11 01:20:27.635174 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:27.635145    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28c374e4-7a51-4884-aa7f-8080b6d30473-plugins-config-volume\") pod \"28c374e4-7a51-4884-aa7f-8080b6d30473\" (UID: \"28c374e4-7a51-4884-aa7f-8080b6d30473\") "
Sep 11 01:20:27.635333 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:27.635305    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45a3629e-56a1-41b3-892f-8b2d7269e996-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "45a3629e-56a1-41b3-892f-8b2d7269e996" (UID: "45a3629e-56a1-41b3-892f-8b2d7269e996"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:20:27.635480 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:27.635462    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/45a3629e-56a1-41b3-892f-8b2d7269e996-plugins-config-volume\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:20:27.635584 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:27.635554    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28c374e4-7a51-4884-aa7f-8080b6d30473-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "28c374e4-7a51-4884-aa7f-8080b6d30473" (UID: "28c374e4-7a51-4884-aa7f-8080b6d30473"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:20:27.637369 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:27.637341    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45a3629e-56a1-41b3-892f-8b2d7269e996-kube-api-access-gfg24" (OuterVolumeSpecName: "kube-api-access-gfg24") pod "45a3629e-56a1-41b3-892f-8b2d7269e996" (UID: "45a3629e-56a1-41b3-892f-8b2d7269e996"). InnerVolumeSpecName "kube-api-access-gfg24". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:20:27.637742 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:27.637726    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28c374e4-7a51-4884-aa7f-8080b6d30473-kube-api-access-tlb7q" (OuterVolumeSpecName: "kube-api-access-tlb7q") pod "28c374e4-7a51-4884-aa7f-8080b6d30473" (UID: "28c374e4-7a51-4884-aa7f-8080b6d30473"). InnerVolumeSpecName "kube-api-access-tlb7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:20:27.736518 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:27.736491    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28c374e4-7a51-4884-aa7f-8080b6d30473-plugins-config-volume\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:20:27.736518 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:27.736514    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfg24\" (UniqueName: \"kubernetes.io/projected/45a3629e-56a1-41b3-892f-8b2d7269e996-kube-api-access-gfg24\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:20:27.736668 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:27.736524    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlb7q\" (UniqueName: \"kubernetes.io/projected/28c374e4-7a51-4884-aa7f-8080b6d30473-kube-api-access-tlb7q\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:20:27.867397 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:27.867328    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:20:28.577601 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:28.577571    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e98d1e-98bcf4-brhgp"
Sep 11 01:20:28.578058 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:28.577570    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e98d1e-98bcf4-brhgp" event={"ID":"28c374e4-7a51-4884-aa7f-8080b6d30473","Type":"ContainerDied","Data":"7c6701e0c8712fda0d36da089a747f9af5ac9ed8626f99095b6c3ae4c90705c7"}
Sep 11 01:20:28.578058 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:28.577720    2574 scope.go:117] "RemoveContainer" containerID="78c81e3c5ff97d6813c3516509b7aac909071720a22d48068957f5a06d32f2bc"
Sep 11 01:20:28.579193 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:28.579136    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e98d1e-65864c5fb5-ggszd"
Sep 11 01:20:28.579193 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:28.579180    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e98d1e-65864c5fb5-ggszd" event={"ID":"45a3629e-56a1-41b3-892f-8b2d7269e996","Type":"ContainerDied","Data":"f4e3632b40092821fd1224c86619c29a348ba047216f25904c12d8b285099b78"}
Sep 11 01:20:28.587377 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:28.587359    2574 scope.go:117] "RemoveContainer" containerID="fb01b84ebb7eaeb9c459aca71eb842c7c6707ed4e13f86c55e4aed3cd0d6ec0c"
Sep 11 01:20:28.601120 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:28.601095    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e98d1e-98bcf4-brhgp"]
Sep 11 01:20:28.604443 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:28.604423    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e98d1e-98bcf4-brhgp"]
Sep 11 01:20:28.615739 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:28.615716    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e98d1e-65864c5fb5-ggszd"]
Sep 11 01:20:28.625794 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:28.625773    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e98d1e-65864c5fb5-ggszd"]
Sep 11 01:20:29.100953 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:29.100893    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28c374e4-7a51-4884-aa7f-8080b6d30473" path="/var/lib/kubelet/pods/28c374e4-7a51-4884-aa7f-8080b6d30473/volumes"
Sep 11 01:20:29.107602 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:29.101788    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45a3629e-56a1-41b3-892f-8b2d7269e996" path="/var/lib/kubelet/pods/45a3629e-56a1-41b3-892f-8b2d7269e996/volumes"
Sep 11 01:20:29.107602 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:29.102347    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7dea1f0a-45c3-4a3d-b84c-ae450612c7de" path="/var/lib/kubelet/pods/7dea1f0a-45c3-4a3d-b84c-ae450612c7de/volumes"
Sep 11 01:20:31.498713 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:31.498662    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698" podUID="ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f" containerName="main" containerID="cri-o://3444daec1d551076df1edef44a36da522126a0d8d18310b250ddf078e7e2c7bb" gracePeriod=45
Sep 11 01:20:31.757285 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:31.757225    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698"
Sep 11 01:20:31.868749 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:31.868716    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-tmp-dir\") pod \"ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f\" (UID: \"ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f\") "
Sep 11 01:20:31.868969 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:31.868763    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-home\") pod \"ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f\" (UID: \"ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f\") "
Sep 11 01:20:31.868969 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:31.868787    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-tls-certs\") pod \"ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f\" (UID: \"ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f\") "
Sep 11 01:20:31.868969 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:31.868848    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-dshm\") pod \"ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f\" (UID: \"ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f\") "
Sep 11 01:20:31.868969 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:31.868886    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-model-cache\") pod \"ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f\" (UID: \"ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f\") "
Sep 11 01:20:31.868969 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:31.868960    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-972tn\" (UniqueName: \"kubernetes.io/projected/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-kube-api-access-972tn\") pod \"ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f\" (UID: \"ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f\") "
Sep 11 01:20:31.869227 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:31.869030    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f" (UID: "ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:20:31.869227 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:31.869091    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-home" (OuterVolumeSpecName: "home") pod "ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f" (UID: "ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:20:31.869295 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:31.869218    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-model-cache" (OuterVolumeSpecName: "model-cache") pod "ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f" (UID: "ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:20:31.869295 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:31.869287    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-tmp-dir\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:20:31.869358 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:31.869302    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-home\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:20:31.871278 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:31.871247    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-dshm" (OuterVolumeSpecName: "dshm") pod "ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f" (UID: "ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:20:31.871278 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:31.871260    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-kube-api-access-972tn" (OuterVolumeSpecName: "kube-api-access-972tn") pod "ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f" (UID: "ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f"). InnerVolumeSpecName "kube-api-access-972tn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:20:31.871426 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:31.871357    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f" (UID: "ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:20:31.970587 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:31.970534    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-dshm\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:20:31.970587 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:31.970579    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-model-cache\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:20:31.970587 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:31.970590    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-972tn\" (UniqueName: \"kubernetes.io/projected/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-kube-api-access-972tn\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:20:31.970587 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:31.970600    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f-tls-certs\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:20:32.606183 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:32.606150    2574 generic.go:356] "Generic (PLEG): container finished" podID="ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f" containerID="3444daec1d551076df1edef44a36da522126a0d8d18310b250ddf078e7e2c7bb" exitCode=0
Sep 11 01:20:32.606627 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:32.606190    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698"
Sep 11 01:20:32.606627 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:32.606235    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698" event={"ID":"ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f","Type":"ContainerDied","Data":"3444daec1d551076df1edef44a36da522126a0d8d18310b250ddf078e7e2c7bb"}
Sep 11 01:20:32.606627 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:32.606274    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698" event={"ID":"ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f","Type":"ContainerDied","Data":"7e36aaf43557494f24689603ee544531176ad9123c6339e6c67599f862c07119"}
Sep 11 01:20:32.606627 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:32.606291    2574 scope.go:117] "RemoveContainer" containerID="3444daec1d551076df1edef44a36da522126a0d8d18310b250ddf078e7e2c7bb"
Sep 11 01:20:32.615747 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:32.615731    2574 scope.go:117] "RemoveContainer" containerID="3444daec1d551076df1edef44a36da522126a0d8d18310b250ddf078e7e2c7bb"
Sep 11 01:20:32.616127 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:20:32.616108    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3444daec1d551076df1edef44a36da522126a0d8d18310b250ddf078e7e2c7bb\": container with ID starting with 3444daec1d551076df1edef44a36da522126a0d8d18310b250ddf078e7e2c7bb not found: ID does not exist" containerID="3444daec1d551076df1edef44a36da522126a0d8d18310b250ddf078e7e2c7bb"
Sep 11 01:20:32.616183 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:32.616135    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3444daec1d551076df1edef44a36da522126a0d8d18310b250ddf078e7e2c7bb"} err="failed to get container status \"3444daec1d551076df1edef44a36da522126a0d8d18310b250ddf078e7e2c7bb\": rpc error: code = NotFound desc = could not find container \"3444daec1d551076df1edef44a36da522126a0d8d18310b250ddf078e7e2c7bb\": container with ID starting with 3444daec1d551076df1edef44a36da522126a0d8d18310b250ddf078e7e2c7bb not found: ID does not exist"
Sep 11 01:20:32.626574 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:32.626551    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698"]
Sep 11 01:20:32.628700 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:32.628678    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-10a215/test-model-10a215-kserve-7df754df8f-tn698"]
Sep 11 01:20:32.852988 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:32.852955    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf"]
Sep 11 01:20:32.949561 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:32.949530    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:20:33.101164 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:33.101131    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f" path="/var/lib/kubelet/pods/ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f/volumes"
Sep 11 01:20:47.872672 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:47.872607    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf" podUID="effe0140-b5e3-40bc-856b-d620885a61a1" containerName="main" containerID="cri-o://f1407ab9b8d43fbb0fd863efb01f47ed665e9645ea8211f1d11c3138cf9a7d34" gracePeriod=45
Sep 11 01:20:48.125083 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:48.125020    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf"
Sep 11 01:20:48.218136 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:48.218108    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2glqt\" (UniqueName: \"kubernetes.io/projected/effe0140-b5e3-40bc-856b-d620885a61a1-kube-api-access-2glqt\") pod \"effe0140-b5e3-40bc-856b-d620885a61a1\" (UID: \"effe0140-b5e3-40bc-856b-d620885a61a1\") "
Sep 11 01:20:48.218303 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:48.218145    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/effe0140-b5e3-40bc-856b-d620885a61a1-tmp-dir\") pod \"effe0140-b5e3-40bc-856b-d620885a61a1\" (UID: \"effe0140-b5e3-40bc-856b-d620885a61a1\") "
Sep 11 01:20:48.218303 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:48.218176    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/effe0140-b5e3-40bc-856b-d620885a61a1-model-cache\") pod \"effe0140-b5e3-40bc-856b-d620885a61a1\" (UID: \"effe0140-b5e3-40bc-856b-d620885a61a1\") "
Sep 11 01:20:48.218303 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:48.218251    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/effe0140-b5e3-40bc-856b-d620885a61a1-tls-certs\") pod \"effe0140-b5e3-40bc-856b-d620885a61a1\" (UID: \"effe0140-b5e3-40bc-856b-d620885a61a1\") "
Sep 11 01:20:48.218303 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:48.218295    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/effe0140-b5e3-40bc-856b-d620885a61a1-home\") pod \"effe0140-b5e3-40bc-856b-d620885a61a1\" (UID: \"effe0140-b5e3-40bc-856b-d620885a61a1\") "
Sep 11 01:20:48.218496 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:48.218354    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/effe0140-b5e3-40bc-856b-d620885a61a1-dshm\") pod \"effe0140-b5e3-40bc-856b-d620885a61a1\" (UID: \"effe0140-b5e3-40bc-856b-d620885a61a1\") "
Sep 11 01:20:48.218496 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:48.218418    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/effe0140-b5e3-40bc-856b-d620885a61a1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "effe0140-b5e3-40bc-856b-d620885a61a1" (UID: "effe0140-b5e3-40bc-856b-d620885a61a1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:20:48.218496 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:48.218424    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/effe0140-b5e3-40bc-856b-d620885a61a1-model-cache" (OuterVolumeSpecName: "model-cache") pod "effe0140-b5e3-40bc-856b-d620885a61a1" (UID: "effe0140-b5e3-40bc-856b-d620885a61a1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:20:48.218656 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:48.218626    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/effe0140-b5e3-40bc-856b-d620885a61a1-home" (OuterVolumeSpecName: "home") pod "effe0140-b5e3-40bc-856b-d620885a61a1" (UID: "effe0140-b5e3-40bc-856b-d620885a61a1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:20:48.218736 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:48.218720    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/effe0140-b5e3-40bc-856b-d620885a61a1-tmp-dir\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:20:48.218792 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:48.218739    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/effe0140-b5e3-40bc-856b-d620885a61a1-model-cache\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:20:48.218792 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:48.218747    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/effe0140-b5e3-40bc-856b-d620885a61a1-home\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:20:48.220509 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:48.220485    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/effe0140-b5e3-40bc-856b-d620885a61a1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "effe0140-b5e3-40bc-856b-d620885a61a1" (UID: "effe0140-b5e3-40bc-856b-d620885a61a1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:20:48.220617 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:48.220570    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/effe0140-b5e3-40bc-856b-d620885a61a1-kube-api-access-2glqt" (OuterVolumeSpecName: "kube-api-access-2glqt") pod "effe0140-b5e3-40bc-856b-d620885a61a1" (UID: "effe0140-b5e3-40bc-856b-d620885a61a1"). InnerVolumeSpecName "kube-api-access-2glqt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:20:48.220617 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:48.220578    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/effe0140-b5e3-40bc-856b-d620885a61a1-dshm" (OuterVolumeSpecName: "dshm") pod "effe0140-b5e3-40bc-856b-d620885a61a1" (UID: "effe0140-b5e3-40bc-856b-d620885a61a1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:20:48.319210 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:48.319187    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/effe0140-b5e3-40bc-856b-d620885a61a1-dshm\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:20:48.319327 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:48.319234    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2glqt\" (UniqueName: \"kubernetes.io/projected/effe0140-b5e3-40bc-856b-d620885a61a1-kube-api-access-2glqt\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:20:48.319327 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:48.319246    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/effe0140-b5e3-40bc-856b-d620885a61a1-tls-certs\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:20:48.694662 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:48.694625    2574 generic.go:356] "Generic (PLEG): container finished" podID="effe0140-b5e3-40bc-856b-d620885a61a1" containerID="f1407ab9b8d43fbb0fd863efb01f47ed665e9645ea8211f1d11c3138cf9a7d34" exitCode=0
Sep 11 01:20:48.694860 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:48.694676    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf"
Sep 11 01:20:48.694860 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:48.694715    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf" event={"ID":"effe0140-b5e3-40bc-856b-d620885a61a1","Type":"ContainerDied","Data":"f1407ab9b8d43fbb0fd863efb01f47ed665e9645ea8211f1d11c3138cf9a7d34"}
Sep 11 01:20:48.694860 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:48.694765    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf" event={"ID":"effe0140-b5e3-40bc-856b-d620885a61a1","Type":"ContainerDied","Data":"e6da470e2e3b5e84bacd54d049484ccd48d040a0f76ba9c9739972ba860a9445"}
Sep 11 01:20:48.694860 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:48.694786    2574 scope.go:117] "RemoveContainer" containerID="f1407ab9b8d43fbb0fd863efb01f47ed665e9645ea8211f1d11c3138cf9a7d34"
Sep 11 01:20:48.703880 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:48.703865    2574 scope.go:117] "RemoveContainer" containerID="f1407ab9b8d43fbb0fd863efb01f47ed665e9645ea8211f1d11c3138cf9a7d34"
Sep 11 01:20:48.704141 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:20:48.704123    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1407ab9b8d43fbb0fd863efb01f47ed665e9645ea8211f1d11c3138cf9a7d34\": container with ID starting with f1407ab9b8d43fbb0fd863efb01f47ed665e9645ea8211f1d11c3138cf9a7d34 not found: ID does not exist" containerID="f1407ab9b8d43fbb0fd863efb01f47ed665e9645ea8211f1d11c3138cf9a7d34"
Sep 11 01:20:48.704189 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:48.704147    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1407ab9b8d43fbb0fd863efb01f47ed665e9645ea8211f1d11c3138cf9a7d34"} err="failed to get container status \"f1407ab9b8d43fbb0fd863efb01f47ed665e9645ea8211f1d11c3138cf9a7d34\": rpc error: code = NotFound desc = could not find container \"f1407ab9b8d43fbb0fd863efb01f47ed665e9645ea8211f1d11c3138cf9a7d34\": container with ID starting with f1407ab9b8d43fbb0fd863efb01f47ed665e9645ea8211f1d11c3138cf9a7d34 not found: ID does not exist"
Sep 11 01:20:48.716351 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:48.716327    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf"]
Sep 11 01:20:48.720688 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:48.720666    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-e98d1e/test-model-e98d1e-kserve-6454d8c4df-dwccf"]
Sep 11 01:20:49.100165 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:49.100084    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="effe0140-b5e3-40bc-856b-d620885a61a1" path="/var/lib/kubelet/pods/effe0140-b5e3-40bc-856b-d620885a61a1/volumes"
Sep 11 01:20:55.622946 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.622894    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:20:55.623440 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.623025    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6"]
Sep 11 01:20:55.623440 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.623404    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f" containerName="main"
Sep 11 01:20:55.623440 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.623415    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f" containerName="main"
Sep 11 01:20:55.623440 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.623433    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28c374e4-7a51-4884-aa7f-8080b6d30473" containerName="payload-pre-processing"
Sep 11 01:20:55.623440 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.623438    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="28c374e4-7a51-4884-aa7f-8080b6d30473" containerName="payload-pre-processing"
Sep 11 01:20:55.623624 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.623449    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45a3629e-56a1-41b3-892f-8b2d7269e996" containerName="payload-processing"
Sep 11 01:20:55.623624 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.623455    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="45a3629e-56a1-41b3-892f-8b2d7269e996" containerName="payload-processing"
Sep 11 01:20:55.623624 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.623461    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7dea1f0a-45c3-4a3d-b84c-ae450612c7de" containerName="revoke-keys"
Sep 11 01:20:55.623624 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.623466    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7dea1f0a-45c3-4a3d-b84c-ae450612c7de" containerName="revoke-keys"
Sep 11 01:20:55.623624 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.623474    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="effe0140-b5e3-40bc-856b-d620885a61a1" containerName="main"
Sep 11 01:20:55.623624 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.623479    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="effe0140-b5e3-40bc-856b-d620885a61a1" containerName="main"
Sep 11 01:20:55.623624 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.623533    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="28c374e4-7a51-4884-aa7f-8080b6d30473" containerName="payload-pre-processing"
Sep 11 01:20:55.623624 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.623542    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebf1d8a3-5b6c-4684-a95f-d5d0be6c571f" containerName="main"
Sep 11 01:20:55.623624 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.623548    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7dea1f0a-45c3-4a3d-b84c-ae450612c7de" containerName="revoke-keys"
Sep 11 01:20:55.623624 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.623555    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="effe0140-b5e3-40bc-856b-d620885a61a1" containerName="main"
Sep 11 01:20:55.623624 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.623561    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="45a3629e-56a1-41b3-892f-8b2d7269e996" containerName="payload-processing"
Sep 11 01:20:55.627993 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.627975    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6"
Sep 11 01:20:55.631077 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.631047    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6f69f6e6\"/\"openshift-service-ca.crt\""
Sep 11 01:20:55.631195 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.631055    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6f69f6e6\"/\"kube-root-ca.crt\""
Sep 11 01:20:55.631195 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.631055    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6f69f6e6\"/\"default-dockercfg-kjcnd\""
Sep 11 01:20:55.631195 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.631164    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6f69f6e6\"/\"e2e-lifecycle-model-6f69f6e6-kserve-self-signed-certs\""
Sep 11 01:20:55.636029 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.636010    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6"]
Sep 11 01:20:55.784349 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.784320    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-model-cache\") pod \"e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6\" (UID: \"fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc\") " pod="ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6"
Sep 11 01:20:55.784349 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.784359    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-home\") pod \"e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6\" (UID: \"fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc\") " pod="ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6"
Sep 11 01:20:55.784572 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.784376    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-tmp-dir\") pod \"e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6\" (UID: \"fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc\") " pod="ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6"
Sep 11 01:20:55.784572 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.784486    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-tls-certs\") pod \"e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6\" (UID: \"fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc\") " pod="ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6"
Sep 11 01:20:55.784572 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.784532    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmkd8\" (UniqueName: \"kubernetes.io/projected/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-kube-api-access-bmkd8\") pod \"e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6\" (UID: \"fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc\") " pod="ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6"
Sep 11 01:20:55.784688 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.784576    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-dshm\") pod \"e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6\" (UID: \"fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc\") " pod="ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6"
Sep 11 01:20:55.885145 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.885073    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-home\") pod \"e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6\" (UID: \"fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc\") " pod="ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6"
Sep 11 01:20:55.885145 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.885114    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-tmp-dir\") pod \"e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6\" (UID: \"fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc\") " pod="ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6"
Sep 11 01:20:55.885337 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.885171    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-tls-certs\") pod \"e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6\" (UID: \"fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc\") " pod="ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6"
Sep 11 01:20:55.885337 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.885218    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmkd8\" (UniqueName: \"kubernetes.io/projected/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-kube-api-access-bmkd8\") pod \"e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6\" (UID: \"fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc\") " pod="ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6"
Sep 11 01:20:55.885337 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.885257    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-dshm\") pod \"e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6\" (UID: \"fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc\") " pod="ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6"
Sep 11 01:20:55.885337 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.885316    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-model-cache\") pod \"e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6\" (UID: \"fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc\") " pod="ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6"
Sep 11 01:20:55.885536 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.885494    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-home\") pod \"e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6\" (UID: \"fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc\") " pod="ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6"
Sep 11 01:20:55.885634 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.885605    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-tmp-dir\") pod \"e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6\" (UID: \"fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc\") " pod="ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6"
Sep 11 01:20:55.885742 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.885641    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-model-cache\") pod \"e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6\" (UID: \"fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc\") " pod="ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6"
Sep 11 01:20:55.887569 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.887548    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-dshm\") pod \"e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6\" (UID: \"fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc\") " pod="ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6"
Sep 11 01:20:55.887848 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.887830    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-tls-certs\") pod \"e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6\" (UID: \"fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc\") " pod="ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6"
Sep 11 01:20:55.895247 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.895211    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmkd8\" (UniqueName: \"kubernetes.io/projected/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-kube-api-access-bmkd8\") pod \"e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6\" (UID: \"fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc\") " pod="ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6"
Sep 11 01:20:55.937339 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:55.937304    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6"
Sep 11 01:20:56.214658 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:56.214622    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6"]
Sep 11 01:20:56.217500 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:20:56.217477    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfbf3e0de_16d4_4c8e_ab36_4d2ae019b3bc.slice/crio-48e5b9b20558c4fa2fa2cca19e83bfefa7b23217a2b81bc162fa722da9d1ac39 WatchSource:0}: Error finding container 48e5b9b20558c4fa2fa2cca19e83bfefa7b23217a2b81bc162fa722da9d1ac39: Status 404 returned error can't find the container with id 48e5b9b20558c4fa2fa2cca19e83bfefa7b23217a2b81bc162fa722da9d1ac39
Sep 11 01:20:56.739663 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:56.739629    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6" event={"ID":"fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc","Type":"ContainerStarted","Data":"15b0990397ed15ae482e675f0948ead9fb7c39eb201b89a945d479efb61130e8"}
Sep 11 01:20:56.740174 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:56.739671    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6" event={"ID":"fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc","Type":"ContainerStarted","Data":"48e5b9b20558c4fa2fa2cca19e83bfefa7b23217a2b81bc162fa722da9d1ac39"}
Sep 11 01:20:56.756549 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:56.756506    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6" podStartSLOduration=1.756491764 podStartE2EDuration="1.756491764s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:20:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:20:56.754652113 +0000 UTC m=+1372.151844287" watchObservedRunningTime="2026-09-11 01:20:56.756491764 +0000 UTC m=+1372.153683937"
Sep 11 01:20:57.973149 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:57.973119    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:20:58.204956 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:58.204904    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:20:58.343510 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:58.343438    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:20:59.193187 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:20:59.193148    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:21:03.353188 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:03.353121    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:21:05.325168 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:05.325137    2574 scope.go:117] "RemoveContainer" containerID="45447558de0135b18547fd79fe712a7153f5144290d638d812ad28ca9d46403d"
Sep 11 01:21:05.333341 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:05.333320    2574 scope.go:117] "RemoveContainer" containerID="24d1603ef3528d23f1130776a376fc9b78452dbca43889e0d4eb9776845c0024"
Sep 11 01:21:05.340744 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:05.340724    2574 scope.go:117] "RemoveContainer" containerID="fa23161924d23c7bf0cdf2fef18cf161f1d491389e406605121db74c0048b856"
Sep 11 01:21:05.348208 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:05.348192    2574 scope.go:117] "RemoveContainer" containerID="add88a5f21591f3a0ae63a12349b0b30e565262bd890ca9e5d42857e2b4282e2"
Sep 11 01:21:05.937669 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:05.937635    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6"
Sep 11 01:21:05.937867 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:05.937709    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6"
Sep 11 01:21:05.950776 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:05.950748    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6"
Sep 11 01:21:06.806835 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:06.806806    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6"
Sep 11 01:21:09.562718 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:09.562681    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:21:09.834534 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:09.834456    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:21:10.164357 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:10.164326    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-10a215-6jkh7"]
Sep 11 01:21:10.169753 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:10.169730    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-10a215-6jkh7"
Sep 11 01:21:10.172681 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:10.172654    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 01:21:10.172934 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:10.172687    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q6gbn\""
Sep 11 01:21:10.174880 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:10.174849    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-10a215-6jkh7"]
Sep 11 01:21:10.212506 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:10.212472    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcbadff2-716f-44c8-8c47-cc648f6396fb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-10a215-6jkh7\" (UID: \"dcbadff2-716f-44c8-8c47-cc648f6396fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-10a215-6jkh7"
Sep 11 01:21:10.275926 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:10.275876    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:21:10.313558 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:10.313524    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcbadff2-716f-44c8-8c47-cc648f6396fb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-10a215-6jkh7\" (UID: \"dcbadff2-716f-44c8-8c47-cc648f6396fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-10a215-6jkh7"
Sep 11 01:21:10.314134 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:10.314113    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcbadff2-716f-44c8-8c47-cc648f6396fb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-10a215-6jkh7\" (UID: \"dcbadff2-716f-44c8-8c47-cc648f6396fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-10a215-6jkh7"
Sep 11 01:21:10.484016 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:10.483899    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-10a215-6jkh7"
Sep 11 01:21:10.759733 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:10.759708    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-10a215-6jkh7"]
Sep 11 01:21:10.762088 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:21:10.762059    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcbadff2_716f_44c8_8c47_cc648f6396fb.slice/crio-78aedf675628b90d707ff2bd6fb043cad9265dfd85c6d04479fa4b3fbdc6b12e WatchSource:0}: Error finding container 78aedf675628b90d707ff2bd6fb043cad9265dfd85c6d04479fa4b3fbdc6b12e: Status 404 returned error can't find the container with id 78aedf675628b90d707ff2bd6fb043cad9265dfd85c6d04479fa4b3fbdc6b12e
Sep 11 01:21:10.818469 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:10.818426    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-10a215-6jkh7" event={"ID":"dcbadff2-716f-44c8-8c47-cc648f6396fb","Type":"ContainerStarted","Data":"78aedf675628b90d707ff2bd6fb043cad9265dfd85c6d04479fa4b3fbdc6b12e"}
Sep 11 01:21:11.825208 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:11.825170    2574 generic.go:356] "Generic (PLEG): container finished" podID="dcbadff2-716f-44c8-8c47-cc648f6396fb" containerID="972ea5ada54b92b45b1dd06904e174b003e0a5fa85514437888d9bbf0f086567" exitCode=0
Sep 11 01:21:11.825577 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:11.825237    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-10a215-6jkh7" event={"ID":"dcbadff2-716f-44c8-8c47-cc648f6396fb","Type":"ContainerDied","Data":"972ea5ada54b92b45b1dd06904e174b003e0a5fa85514437888d9bbf0f086567"}
Sep 11 01:21:12.960419 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:12.960394    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-10a215-6jkh7"
Sep 11 01:21:13.036452 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:13.036408    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcbadff2-716f-44c8-8c47-cc648f6396fb-maas-api-service-ca\") pod \"dcbadff2-716f-44c8-8c47-cc648f6396fb\" (UID: \"dcbadff2-716f-44c8-8c47-cc648f6396fb\") "
Sep 11 01:21:13.036729 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:13.036708    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dcbadff2-716f-44c8-8c47-cc648f6396fb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dcbadff2-716f-44c8-8c47-cc648f6396fb" (UID: "dcbadff2-716f-44c8-8c47-cc648f6396fb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:21:13.137967 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:13.137935    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcbadff2-716f-44c8-8c47-cc648f6396fb-maas-api-service-ca\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:21:13.837564 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:13.837536    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-10a215-6jkh7"
Sep 11 01:21:13.837756 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:13.837547    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-10a215-6jkh7" event={"ID":"dcbadff2-716f-44c8-8c47-cc648f6396fb","Type":"ContainerDied","Data":"78aedf675628b90d707ff2bd6fb043cad9265dfd85c6d04479fa4b3fbdc6b12e"}
Sep 11 01:21:13.837756 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:13.837652    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="78aedf675628b90d707ff2bd6fb043cad9265dfd85c6d04479fa4b3fbdc6b12e"
Sep 11 01:21:20.657477 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:20.657444    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-10a215-29818155-6mcdd"]
Sep 11 01:21:20.660113 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:20.660086    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-10a215-29818155-6mcdd"]
Sep 11 01:21:20.660476 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:20.660456    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:21:20.685381 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:20.685346    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-10a215-6bf5478bc4-z948j"]
Sep 11 01:21:20.685619 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:20.685578    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-10a215-6bf5478bc4-z948j" podUID="7c5b6724-374f-492d-a3e3-e1ce56905d8b" containerName="payload-pre-processing" containerID="cri-o://82f908c4cf47c74206563f00e9431f81b2ad29dab9d3fb11b691d497a0d686d2" gracePeriod=30
Sep 11 01:21:20.727976 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:20.727944    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:21:20.828241 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:20.828203    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:21:20.845904 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:20.845871    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-10a215-6jkh7"]
Sep 11 01:21:20.848111 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:20.848091    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-10a215-6jkh7"]
Sep 11 01:21:20.882214 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:20.882186    2574 generic.go:356] "Generic (PLEG): container finished" podID="7c5b6724-374f-492d-a3e3-e1ce56905d8b" containerID="82f908c4cf47c74206563f00e9431f81b2ad29dab9d3fb11b691d497a0d686d2" exitCode=0
Sep 11 01:21:20.882394 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:20.882262    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-10a215-6bf5478bc4-z948j" event={"ID":"7c5b6724-374f-492d-a3e3-e1ce56905d8b","Type":"ContainerDied","Data":"82f908c4cf47c74206563f00e9431f81b2ad29dab9d3fb11b691d497a0d686d2"}
Sep 11 01:21:20.909005 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:20.908978    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:21:20.938959 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:20.938933    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-10a215-6bf5478bc4-z948j"
Sep 11 01:21:21.006143 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:21.006112    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42h75\" (UniqueName: \"kubernetes.io/projected/7c5b6724-374f-492d-a3e3-e1ce56905d8b-kube-api-access-42h75\") pod \"7c5b6724-374f-492d-a3e3-e1ce56905d8b\" (UID: \"7c5b6724-374f-492d-a3e3-e1ce56905d8b\") "
Sep 11 01:21:21.006308 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:21.006161    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c5b6724-374f-492d-a3e3-e1ce56905d8b-plugins-config-volume\") pod \"7c5b6724-374f-492d-a3e3-e1ce56905d8b\" (UID: \"7c5b6724-374f-492d-a3e3-e1ce56905d8b\") "
Sep 11 01:21:21.006577 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:21.006551    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c5b6724-374f-492d-a3e3-e1ce56905d8b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7c5b6724-374f-492d-a3e3-e1ce56905d8b" (UID: "7c5b6724-374f-492d-a3e3-e1ce56905d8b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:21:21.008406 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:21.008379    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c5b6724-374f-492d-a3e3-e1ce56905d8b-kube-api-access-42h75" (OuterVolumeSpecName: "kube-api-access-42h75") pod "7c5b6724-374f-492d-a3e3-e1ce56905d8b" (UID: "7c5b6724-374f-492d-a3e3-e1ce56905d8b"). InnerVolumeSpecName "kube-api-access-42h75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:21:21.106012 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:21.105979    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a48094a-84dd-4058-b886-f7a2fd9111b9" path="/var/lib/kubelet/pods/7a48094a-84dd-4058-b886-f7a2fd9111b9/volumes"
Sep 11 01:21:21.106307 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:21.106293    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dcbadff2-716f-44c8-8c47-cc648f6396fb" path="/var/lib/kubelet/pods/dcbadff2-716f-44c8-8c47-cc648f6396fb/volumes"
Sep 11 01:21:21.107050 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:21.107031    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42h75\" (UniqueName: \"kubernetes.io/projected/7c5b6724-374f-492d-a3e3-e1ce56905d8b-kube-api-access-42h75\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:21:21.107099 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:21.107053    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c5b6724-374f-492d-a3e3-e1ce56905d8b-plugins-config-volume\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:21:21.212176 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:21.212085    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:21:21.889092 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:21.889059    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-10a215-6bf5478bc4-z948j"
Sep 11 01:21:21.889092 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:21.889080    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-10a215-6bf5478bc4-z948j" event={"ID":"7c5b6724-374f-492d-a3e3-e1ce56905d8b","Type":"ContainerDied","Data":"0ea1c717682c13954058a2b9a73fa6d007c60e4cb1dcb7ed27b8fa9a00944caf"}
Sep 11 01:21:21.889564 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:21.889138    2574 scope.go:117] "RemoveContainer" containerID="82f908c4cf47c74206563f00e9431f81b2ad29dab9d3fb11b691d497a0d686d2"
Sep 11 01:21:21.906279 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:21.906252    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-10a215-6bf5478bc4-z948j"]
Sep 11 01:21:21.910245 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:21.910219    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-10a215-6bf5478bc4-z948j"]
Sep 11 01:21:23.100799 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:23.100766    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c5b6724-374f-492d-a3e3-e1ce56905d8b" path="/var/lib/kubelet/pods/7c5b6724-374f-492d-a3e3-e1ce56905d8b/volumes"
Sep 11 01:21:27.010621 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:27.010575    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:21:27.110842 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:27.110802    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:21:27.288171 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:27.288091    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:21:27.867965 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:27.867930    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:21:28.271676 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:28.271644    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:21:28.371187 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:28.371151    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:21:31.562530 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:31.562489    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:21:31.627205 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:31.627169    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:21:32.840573 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:32.840542    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:21:32.940347 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:32.940308    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:21:33.310405 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:33.310371    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:21:33.411737 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:33.411708    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:21:33.447391 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:33.447362    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-d6e26b-78d4d6b489-jq8ct"]
Sep 11 01:21:33.447729 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:33.447716    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcbadff2-716f-44c8-8c47-cc648f6396fb" containerName="revoke-keys"
Sep 11 01:21:33.447793 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:33.447730    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcbadff2-716f-44c8-8c47-cc648f6396fb" containerName="revoke-keys"
Sep 11 01:21:33.447793 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:33.447748    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c5b6724-374f-492d-a3e3-e1ce56905d8b" containerName="payload-pre-processing"
Sep 11 01:21:33.447793 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:33.447754    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c5b6724-374f-492d-a3e3-e1ce56905d8b" containerName="payload-pre-processing"
Sep 11 01:21:33.447966 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:33.447817    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcbadff2-716f-44c8-8c47-cc648f6396fb" containerName="revoke-keys"
Sep 11 01:21:33.447966 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:33.447827    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c5b6724-374f-492d-a3e3-e1ce56905d8b" containerName="payload-pre-processing"
Sep 11 01:21:33.454994 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:33.454063    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d6e26b-78d4d6b489-jq8ct"
Sep 11 01:21:33.457570 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:33.457541    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-d6e26b-dockercfg-xgzsb\""
Sep 11 01:21:33.457704 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:33.457573    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-d6e26b\""
Sep 11 01:21:33.461160 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:33.461136    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-d6e26b-78d4d6b489-jq8ct"]
Sep 11 01:21:33.489466 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:33.489442    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d6e26b-64f4dc94fd-ckm8g"]
Sep 11 01:21:33.492817 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:33.492803    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d6e26b-64f4dc94fd-ckm8g"
Sep 11 01:21:33.503488 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:33.503467    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d6e26b-64f4dc94fd-ckm8g"]
Sep 11 01:21:33.518556 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:33.518532    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6frvp\" (UniqueName: \"kubernetes.io/projected/047af868-ffeb-4356-9ed0-3b87b0a548d5-kube-api-access-6frvp\") pod \"payload-processing-e2e-shared-a-w3-d6e26b-78d4d6b489-jq8ct\" (UID: \"047af868-ffeb-4356-9ed0-3b87b0a548d5\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d6e26b-78d4d6b489-jq8ct"
Sep 11 01:21:33.518664 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:33.518575    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/047af868-ffeb-4356-9ed0-3b87b0a548d5-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-d6e26b-78d4d6b489-jq8ct\" (UID: \"047af868-ffeb-4356-9ed0-3b87b0a548d5\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d6e26b-78d4d6b489-jq8ct"
Sep 11 01:21:33.620009 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:33.619941    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/047af868-ffeb-4356-9ed0-3b87b0a548d5-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-d6e26b-78d4d6b489-jq8ct\" (UID: \"047af868-ffeb-4356-9ed0-3b87b0a548d5\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d6e26b-78d4d6b489-jq8ct"
Sep 11 01:21:33.620009 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:33.619999    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw4rx\" (UniqueName: \"kubernetes.io/projected/5f7b932a-27a8-465e-881e-9bb78f4c508b-kube-api-access-rw4rx\") pod \"payload-pre-processing-e2e-shared-a-w3-d6e26b-64f4dc94fd-ckm8g\" (UID: \"5f7b932a-27a8-465e-881e-9bb78f4c508b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d6e26b-64f4dc94fd-ckm8g"
Sep 11 01:21:33.620179 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:33.620031    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f7b932a-27a8-465e-881e-9bb78f4c508b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-d6e26b-64f4dc94fd-ckm8g\" (UID: \"5f7b932a-27a8-465e-881e-9bb78f4c508b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d6e26b-64f4dc94fd-ckm8g"
Sep 11 01:21:33.620179 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:33.620141    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6frvp\" (UniqueName: \"kubernetes.io/projected/047af868-ffeb-4356-9ed0-3b87b0a548d5-kube-api-access-6frvp\") pod \"payload-processing-e2e-shared-a-w3-d6e26b-78d4d6b489-jq8ct\" (UID: \"047af868-ffeb-4356-9ed0-3b87b0a548d5\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d6e26b-78d4d6b489-jq8ct"
Sep 11 01:21:33.620656 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:33.620640    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/047af868-ffeb-4356-9ed0-3b87b0a548d5-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-d6e26b-78d4d6b489-jq8ct\" (UID: \"047af868-ffeb-4356-9ed0-3b87b0a548d5\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d6e26b-78d4d6b489-jq8ct"
Sep 11 01:21:33.627811 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:33.627788    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6frvp\" (UniqueName: \"kubernetes.io/projected/047af868-ffeb-4356-9ed0-3b87b0a548d5-kube-api-access-6frvp\") pod \"payload-processing-e2e-shared-a-w3-d6e26b-78d4d6b489-jq8ct\" (UID: \"047af868-ffeb-4356-9ed0-3b87b0a548d5\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d6e26b-78d4d6b489-jq8ct"
Sep 11 01:21:33.721422 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:33.721394    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw4rx\" (UniqueName: \"kubernetes.io/projected/5f7b932a-27a8-465e-881e-9bb78f4c508b-kube-api-access-rw4rx\") pod \"payload-pre-processing-e2e-shared-a-w3-d6e26b-64f4dc94fd-ckm8g\" (UID: \"5f7b932a-27a8-465e-881e-9bb78f4c508b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d6e26b-64f4dc94fd-ckm8g"
Sep 11 01:21:33.721560 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:33.721426    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f7b932a-27a8-465e-881e-9bb78f4c508b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-d6e26b-64f4dc94fd-ckm8g\" (UID: \"5f7b932a-27a8-465e-881e-9bb78f4c508b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d6e26b-64f4dc94fd-ckm8g"
Sep 11 01:21:33.722086 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:33.722067    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f7b932a-27a8-465e-881e-9bb78f4c508b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-d6e26b-64f4dc94fd-ckm8g\" (UID: \"5f7b932a-27a8-465e-881e-9bb78f4c508b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d6e26b-64f4dc94fd-ckm8g"
Sep 11 01:21:33.734667 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:33.734629    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:21:33.737675 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:33.737648    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw4rx\" (UniqueName: \"kubernetes.io/projected/5f7b932a-27a8-465e-881e-9bb78f4c508b-kube-api-access-rw4rx\") pod \"payload-pre-processing-e2e-shared-a-w3-d6e26b-64f4dc94fd-ckm8g\" (UID: \"5f7b932a-27a8-465e-881e-9bb78f4c508b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d6e26b-64f4dc94fd-ckm8g"
Sep 11 01:21:33.767512 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:33.767487    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d6e26b-78d4d6b489-jq8ct"
Sep 11 01:21:33.802896 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:33.802864    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d6e26b-64f4dc94fd-ckm8g"
Sep 11 01:21:34.045634 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:34.045603    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-d6e26b-78d4d6b489-jq8ct"]
Sep 11 01:21:34.047991 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:21:34.047963    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod047af868_ffeb_4356_9ed0_3b87b0a548d5.slice/crio-3a65bab7c32036cb29637e440f1428d370a9f20f322b8c2b9edec0c10351ecfc WatchSource:0}: Error finding container 3a65bab7c32036cb29637e440f1428d370a9f20f322b8c2b9edec0c10351ecfc: Status 404 returned error can't find the container with id 3a65bab7c32036cb29637e440f1428d370a9f20f322b8c2b9edec0c10351ecfc
Sep 11 01:21:34.088262 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:34.088237    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d6e26b-64f4dc94fd-ckm8g"]
Sep 11 01:21:34.094969 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:21:34.094947    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f7b932a_27a8_465e_881e_9bb78f4c508b.slice/crio-cafe742d1b5cff95b5a9c096e48e1d94aef9ef16ffca8e92cf8535f04e97a452 WatchSource:0}: Error finding container cafe742d1b5cff95b5a9c096e48e1d94aef9ef16ffca8e92cf8535f04e97a452: Status 404 returned error can't find the container with id cafe742d1b5cff95b5a9c096e48e1d94aef9ef16ffca8e92cf8535f04e97a452
Sep 11 01:21:34.960036 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:34.959998    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d6e26b-78d4d6b489-jq8ct" event={"ID":"047af868-ffeb-4356-9ed0-3b87b0a548d5","Type":"ContainerStarted","Data":"aa827dc00c170b65b0d4e242b1d41d4687fbf5975be15cc32c4a3c3b971f20ff"}
Sep 11 01:21:34.960229 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:34.960058    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d6e26b-78d4d6b489-jq8ct"
Sep 11 01:21:34.960229 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:34.960082    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d6e26b-78d4d6b489-jq8ct" event={"ID":"047af868-ffeb-4356-9ed0-3b87b0a548d5","Type":"ContainerStarted","Data":"3a65bab7c32036cb29637e440f1428d370a9f20f322b8c2b9edec0c10351ecfc"}
Sep 11 01:21:34.961508 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:34.961483    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d6e26b-64f4dc94fd-ckm8g" event={"ID":"5f7b932a-27a8-465e-881e-9bb78f4c508b","Type":"ContainerStarted","Data":"00fad2f1502d58b5ca8abdf34535f6c36e6c8aeb8ff98d80d53abf198999a538"}
Sep 11 01:21:34.961645 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:34.961529    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d6e26b-64f4dc94fd-ckm8g" event={"ID":"5f7b932a-27a8-465e-881e-9bb78f4c508b","Type":"ContainerStarted","Data":"cafe742d1b5cff95b5a9c096e48e1d94aef9ef16ffca8e92cf8535f04e97a452"}
Sep 11 01:21:34.961645 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:34.961558    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d6e26b-64f4dc94fd-ckm8g"
Sep 11 01:21:34.976765 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:34.976726    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d6e26b-78d4d6b489-jq8ct" podStartSLOduration=1.9767117239999998 podStartE2EDuration="1.976711724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:21:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:21:34.975758675 +0000 UTC m=+1410.372950849" watchObservedRunningTime="2026-09-11 01:21:34.976711724 +0000 UTC m=+1410.373903898"
Sep 11 01:21:34.992795 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:34.992752    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d6e26b-64f4dc94fd-ckm8g" podStartSLOduration=1.992734192 podStartE2EDuration="1.992734192s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:21:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:21:34.990573666 +0000 UTC m=+1410.387765842" watchObservedRunningTime="2026-09-11 01:21:34.992734192 +0000 UTC m=+1410.389926370"
Sep 11 01:21:39.971097 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:39.971066    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:21:40.386532 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:40.386454    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:21:45.968866 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:45.968833    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d6e26b-64f4dc94fd-ckm8g"
Sep 11 01:21:45.969364 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:45.968903    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d6e26b-78d4d6b489-jq8ct"
Sep 11 01:21:56.433973 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:56.433876    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:21:56.770562 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:56.770477    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:21:57.068280 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:57.068197    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:21:57.359278 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:21:57.359185    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:22:00.750073 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:00.750024    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:22:00.842108 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:00.842078    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:22:01.943406 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:01.943372    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:22:02.053092 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:02.053061    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:22:02.234465 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:02.234370    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:22:02.502226 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:02.502141    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:22:02.608396 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:02.608361    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:22:02.618584 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:02.618555    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b9930f-cb59bb98b-p7ndh"]
Sep 11 01:22:02.622289 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:02.622268    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b9930f-cb59bb98b-p7ndh"
Sep 11 01:22:02.624612 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:02.624587    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-b9930f\""
Sep 11 01:22:02.624750 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:02.624667    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-b9930f-dockercfg-wqwtc\""
Sep 11 01:22:02.631634 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:02.631611    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b9930f-cb59bb98b-p7ndh"]
Sep 11 01:22:02.687504 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:02.687468    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p276m\" (UniqueName: \"kubernetes.io/projected/d5a9e347-6003-4aa2-8025-ffc8823483d0-kube-api-access-p276m\") pod \"payload-pre-processing-e2e-shared-b-w3-b9930f-cb59bb98b-p7ndh\" (UID: \"d5a9e347-6003-4aa2-8025-ffc8823483d0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b9930f-cb59bb98b-p7ndh"
Sep 11 01:22:02.687697 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:02.687515    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5a9e347-6003-4aa2-8025-ffc8823483d0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-b9930f-cb59bb98b-p7ndh\" (UID: \"d5a9e347-6003-4aa2-8025-ffc8823483d0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b9930f-cb59bb98b-p7ndh"
Sep 11 01:22:02.788450 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:02.788349    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p276m\" (UniqueName: \"kubernetes.io/projected/d5a9e347-6003-4aa2-8025-ffc8823483d0-kube-api-access-p276m\") pod \"payload-pre-processing-e2e-shared-b-w3-b9930f-cb59bb98b-p7ndh\" (UID: \"d5a9e347-6003-4aa2-8025-ffc8823483d0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b9930f-cb59bb98b-p7ndh"
Sep 11 01:22:02.788450 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:02.788395    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5a9e347-6003-4aa2-8025-ffc8823483d0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-b9930f-cb59bb98b-p7ndh\" (UID: \"d5a9e347-6003-4aa2-8025-ffc8823483d0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b9930f-cb59bb98b-p7ndh"
Sep 11 01:22:02.789093 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:02.789068    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5a9e347-6003-4aa2-8025-ffc8823483d0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-b9930f-cb59bb98b-p7ndh\" (UID: \"d5a9e347-6003-4aa2-8025-ffc8823483d0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b9930f-cb59bb98b-p7ndh"
Sep 11 01:22:02.799299 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:02.799262    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p276m\" (UniqueName: \"kubernetes.io/projected/d5a9e347-6003-4aa2-8025-ffc8823483d0-kube-api-access-p276m\") pod \"payload-pre-processing-e2e-shared-b-w3-b9930f-cb59bb98b-p7ndh\" (UID: \"d5a9e347-6003-4aa2-8025-ffc8823483d0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b9930f-cb59bb98b-p7ndh"
Sep 11 01:22:02.935652 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:02.935614    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b9930f-cb59bb98b-p7ndh"
Sep 11 01:22:03.207407 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:03.207373    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b9930f-cb59bb98b-p7ndh"]
Sep 11 01:22:03.210837 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:22:03.210806    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5a9e347_6003_4aa2_8025_ffc8823483d0.slice/crio-8dc46c7744dd43bc0f7d756254f62915b6536e157f8ef4760e5990f20259ecb5 WatchSource:0}: Error finding container 8dc46c7744dd43bc0f7d756254f62915b6536e157f8ef4760e5990f20259ecb5: Status 404 returned error can't find the container with id 8dc46c7744dd43bc0f7d756254f62915b6536e157f8ef4760e5990f20259ecb5
Sep 11 01:22:04.115752 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:04.115716    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b9930f-cb59bb98b-p7ndh" event={"ID":"d5a9e347-6003-4aa2-8025-ffc8823483d0","Type":"ContainerStarted","Data":"f7868a051bbb73f57d6724a9d763b81cabbb4040fa7d585f2038caa1a7f10725"}
Sep 11 01:22:04.115965 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:04.115770    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b9930f-cb59bb98b-p7ndh" event={"ID":"d5a9e347-6003-4aa2-8025-ffc8823483d0","Type":"ContainerStarted","Data":"8dc46c7744dd43bc0f7d756254f62915b6536e157f8ef4760e5990f20259ecb5"}
Sep 11 01:22:04.115965 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:04.115829    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b9930f-cb59bb98b-p7ndh"
Sep 11 01:22:04.133987 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:04.133940    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b9930f-cb59bb98b-p7ndh" podStartSLOduration=2.133923771 podStartE2EDuration="2.133923771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:22:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:22:04.1330483 +0000 UTC m=+1439.530240472" watchObservedRunningTime="2026-09-11 01:22:04.133923771 +0000 UTC m=+1439.531115939"
Sep 11 01:22:05.392119 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:05.392082    2574 scope.go:117] "RemoveContainer" containerID="8e183fbd613ade268ad3ff12cb9dbe8b7948b604e47f14ea8ef0f4242842be1d"
Sep 11 01:22:12.630169 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:12.630130    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:22:12.728006 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:12.727973    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:22:15.124846 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:15.124815    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b9930f-cb59bb98b-p7ndh"
Sep 11 01:22:19.257487 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:19.257446    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6f69f6e6-74vm4"]
Sep 11 01:22:19.262725 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:19.262707    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6f69f6e6-74vm4"
Sep 11 01:22:19.265715 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:19.265687    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 01:22:19.265879 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:19.265774    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q6gbn\""
Sep 11 01:22:19.267881 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:19.267860    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6f69f6e6-74vm4"]
Sep 11 01:22:19.347433 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:19.347404    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23140b37-f994-4fc4-8070-aa52821b8b47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6f69f6e6-74vm4\" (UID: \"23140b37-f994-4fc4-8070-aa52821b8b47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6f69f6e6-74vm4"
Sep 11 01:22:19.448675 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:19.448637    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23140b37-f994-4fc4-8070-aa52821b8b47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6f69f6e6-74vm4\" (UID: \"23140b37-f994-4fc4-8070-aa52821b8b47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6f69f6e6-74vm4"
Sep 11 01:22:19.449296 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:19.449276    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23140b37-f994-4fc4-8070-aa52821b8b47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6f69f6e6-74vm4\" (UID: \"23140b37-f994-4fc4-8070-aa52821b8b47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6f69f6e6-74vm4"
Sep 11 01:22:19.573931 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:19.573834    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6f69f6e6-74vm4"
Sep 11 01:22:19.854353 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:19.854326    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6f69f6e6-74vm4"]
Sep 11 01:22:19.857005 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:22:19.856975    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23140b37_f994_4fc4_8070_aa52821b8b47.slice/crio-d94d14cb2cfa1701d086d75350a042aff6e1d5fa6099b646e8e47e3b8791ab6a WatchSource:0}: Error finding container d94d14cb2cfa1701d086d75350a042aff6e1d5fa6099b646e8e47e3b8791ab6a: Status 404 returned error can't find the container with id d94d14cb2cfa1701d086d75350a042aff6e1d5fa6099b646e8e47e3b8791ab6a
Sep 11 01:22:20.202641 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:20.202607    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6f69f6e6-74vm4" event={"ID":"23140b37-f994-4fc4-8070-aa52821b8b47","Type":"ContainerStarted","Data":"afc3595def1c8f42b89012181c1945e10730fe9a525137c33e8ea9ea93449e63"}
Sep 11 01:22:20.202776 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:20.202674    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6f69f6e6-74vm4" event={"ID":"23140b37-f994-4fc4-8070-aa52821b8b47","Type":"ContainerStarted","Data":"d94d14cb2cfa1701d086d75350a042aff6e1d5fa6099b646e8e47e3b8791ab6a"}
Sep 11 01:22:20.216758 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:20.216708    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6f69f6e6-74vm4" podStartSLOduration=1.216687133 podStartE2EDuration="1.216687133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:22:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:22:20.215956398 +0000 UTC m=+1455.613148571" watchObservedRunningTime="2026-09-11 01:22:20.216687133 +0000 UTC m=+1455.613879311"
Sep 11 01:22:20.359978 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:20.359933    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:22:21.209342 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:21.209307    2574 generic.go:356] "Generic (PLEG): container finished" podID="23140b37-f994-4fc4-8070-aa52821b8b47" containerID="afc3595def1c8f42b89012181c1945e10730fe9a525137c33e8ea9ea93449e63" exitCode=0
Sep 11 01:22:21.209514 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:21.209351    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6f69f6e6-74vm4" event={"ID":"23140b37-f994-4fc4-8070-aa52821b8b47","Type":"ContainerDied","Data":"afc3595def1c8f42b89012181c1945e10730fe9a525137c33e8ea9ea93449e63"}
Sep 11 01:22:22.351203 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:22.351180    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6f69f6e6-74vm4"
Sep 11 01:22:22.474860 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:22.474754    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23140b37-f994-4fc4-8070-aa52821b8b47-maas-api-service-ca\") pod \"23140b37-f994-4fc4-8070-aa52821b8b47\" (UID: \"23140b37-f994-4fc4-8070-aa52821b8b47\") "
Sep 11 01:22:22.475149 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:22.475126    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23140b37-f994-4fc4-8070-aa52821b8b47-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "23140b37-f994-4fc4-8070-aa52821b8b47" (UID: "23140b37-f994-4fc4-8070-aa52821b8b47"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:22:22.475251 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:22.475236    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23140b37-f994-4fc4-8070-aa52821b8b47-maas-api-service-ca\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:22:23.220192 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:23.220159    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6f69f6e6-74vm4" event={"ID":"23140b37-f994-4fc4-8070-aa52821b8b47","Type":"ContainerDied","Data":"d94d14cb2cfa1701d086d75350a042aff6e1d5fa6099b646e8e47e3b8791ab6a"}
Sep 11 01:22:23.220192 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:23.220184    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6f69f6e6-74vm4"
Sep 11 01:22:23.220390 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:23.220204    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d94d14cb2cfa1701d086d75350a042aff6e1d5fa6099b646e8e47e3b8791ab6a"
Sep 11 01:22:24.236000 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:24.235874    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:22:25.606586 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:25.606550    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:22:28.388744 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:28.388695    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:22:29.542101 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:29.542059    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:22:30.149213 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:30.149169    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:22:30.188756 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:30.188725    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6f69f6e6-55f997794-7s9fv"]
Sep 11 01:22:30.189017 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:30.188989    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-6f69f6e6-55f997794-7s9fv" podUID="8452ec4c-930e-45e1-824c-7edf9854201c" containerName="payload-pre-processing" containerID="cri-o://3d1fe21743850ed265a305f8aef272ae26fa67986cefa82b5fc6930d411949e7" gracePeriod=30
Sep 11 01:22:30.249259 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:30.249224    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:22:30.350360 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:30.350279    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:22:30.350883 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:30.350708    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6f69f6e6-74vm4"]
Sep 11 01:22:30.352449 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:30.352422    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6f69f6e6-74vm4"]
Sep 11 01:22:30.419365 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:30.419325    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:22:30.450987 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:30.450958    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6f69f6e6-55f997794-7s9fv"
Sep 11 01:22:30.554646 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:30.554594    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8452ec4c-930e-45e1-824c-7edf9854201c-plugins-config-volume\") pod \"8452ec4c-930e-45e1-824c-7edf9854201c\" (UID: \"8452ec4c-930e-45e1-824c-7edf9854201c\") "
Sep 11 01:22:30.555496 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:30.554727    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k88bf\" (UniqueName: \"kubernetes.io/projected/8452ec4c-930e-45e1-824c-7edf9854201c-kube-api-access-k88bf\") pod \"8452ec4c-930e-45e1-824c-7edf9854201c\" (UID: \"8452ec4c-930e-45e1-824c-7edf9854201c\") "
Sep 11 01:22:30.555496 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:30.554983    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8452ec4c-930e-45e1-824c-7edf9854201c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8452ec4c-930e-45e1-824c-7edf9854201c" (UID: "8452ec4c-930e-45e1-824c-7edf9854201c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:22:30.555496 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:30.555143    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8452ec4c-930e-45e1-824c-7edf9854201c-plugins-config-volume\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:22:30.556945 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:30.556925    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8452ec4c-930e-45e1-824c-7edf9854201c-kube-api-access-k88bf" (OuterVolumeSpecName: "kube-api-access-k88bf") pod "8452ec4c-930e-45e1-824c-7edf9854201c" (UID: "8452ec4c-930e-45e1-824c-7edf9854201c"). InnerVolumeSpecName "kube-api-access-k88bf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:22:30.656207 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:30.656169    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k88bf\" (UniqueName: \"kubernetes.io/projected/8452ec4c-930e-45e1-824c-7edf9854201c-kube-api-access-k88bf\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:22:30.735181 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:30.735072    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:22:31.100453 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:31.100371    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23140b37-f994-4fc4-8070-aa52821b8b47" path="/var/lib/kubelet/pods/23140b37-f994-4fc4-8070-aa52821b8b47/volumes"
Sep 11 01:22:31.263363 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:31.263332    2574 generic.go:356] "Generic (PLEG): container finished" podID="8452ec4c-930e-45e1-824c-7edf9854201c" containerID="3d1fe21743850ed265a305f8aef272ae26fa67986cefa82b5fc6930d411949e7" exitCode=0
Sep 11 01:22:31.263363 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:31.263369    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6f69f6e6-55f997794-7s9fv" event={"ID":"8452ec4c-930e-45e1-824c-7edf9854201c","Type":"ContainerDied","Data":"3d1fe21743850ed265a305f8aef272ae26fa67986cefa82b5fc6930d411949e7"}
Sep 11 01:22:31.263574 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:31.263377    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6f69f6e6-55f997794-7s9fv"
Sep 11 01:22:31.263574 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:31.263406    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6f69f6e6-55f997794-7s9fv" event={"ID":"8452ec4c-930e-45e1-824c-7edf9854201c","Type":"ContainerDied","Data":"6967ef434d0daa66b0650e936bed7f312779fcf3f1a469a1866411eedbab4341"}
Sep 11 01:22:31.263574 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:31.263427    2574 scope.go:117] "RemoveContainer" containerID="3d1fe21743850ed265a305f8aef272ae26fa67986cefa82b5fc6930d411949e7"
Sep 11 01:22:31.272521 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:31.272501    2574 scope.go:117] "RemoveContainer" containerID="3d1fe21743850ed265a305f8aef272ae26fa67986cefa82b5fc6930d411949e7"
Sep 11 01:22:31.272799 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:22:31.272779    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d1fe21743850ed265a305f8aef272ae26fa67986cefa82b5fc6930d411949e7\": container with ID starting with 3d1fe21743850ed265a305f8aef272ae26fa67986cefa82b5fc6930d411949e7 not found: ID does not exist" containerID="3d1fe21743850ed265a305f8aef272ae26fa67986cefa82b5fc6930d411949e7"
Sep 11 01:22:31.272858 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:31.272810    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d1fe21743850ed265a305f8aef272ae26fa67986cefa82b5fc6930d411949e7"} err="failed to get container status \"3d1fe21743850ed265a305f8aef272ae26fa67986cefa82b5fc6930d411949e7\": rpc error: code = NotFound desc = could not find container \"3d1fe21743850ed265a305f8aef272ae26fa67986cefa82b5fc6930d411949e7\": container with ID starting with 3d1fe21743850ed265a305f8aef272ae26fa67986cefa82b5fc6930d411949e7 not found: ID does not exist"
Sep 11 01:22:31.280229 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:31.280205    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6f69f6e6-55f997794-7s9fv"]
Sep 11 01:22:31.287669 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:31.287645    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6f69f6e6-55f997794-7s9fv"]
Sep 11 01:22:32.491940 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:32.491893    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:22:32.592293 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:32.592261    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:22:33.100974 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:33.100939    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8452ec4c-930e-45e1-824c-7edf9854201c" path="/var/lib/kubelet/pods/8452ec4c-930e-45e1-824c-7edf9854201c/volumes"
Sep 11 01:22:35.709667 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:35.709627    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6"]
Sep 11 01:22:35.764232 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:35.758635    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:22:42.379284 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:42.379247    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55"]
Sep 11 01:22:42.379711 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:42.379696    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23140b37-f994-4fc4-8070-aa52821b8b47" containerName="revoke-keys"
Sep 11 01:22:42.379760 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:42.379712    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="23140b37-f994-4fc4-8070-aa52821b8b47" containerName="revoke-keys"
Sep 11 01:22:42.379760 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:42.379726    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8452ec4c-930e-45e1-824c-7edf9854201c" containerName="payload-pre-processing"
Sep 11 01:22:42.379760 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:42.379731    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8452ec4c-930e-45e1-824c-7edf9854201c" containerName="payload-pre-processing"
Sep 11 01:22:42.379864 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:42.379790    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8452ec4c-930e-45e1-824c-7edf9854201c" containerName="payload-pre-processing"
Sep 11 01:22:42.379864 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:42.379802    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="23140b37-f994-4fc4-8070-aa52821b8b47" containerName="revoke-keys"
Sep 11 01:22:42.382878 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:42.382859    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55"
Sep 11 01:22:42.385737 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:42.385714    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-d6e26b\"/\"auth-test-model-d6e26b-kserve-self-signed-certs\""
Sep 11 01:22:42.385895 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:42.385717    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-d6e26b\"/\"openshift-service-ca.crt\""
Sep 11 01:22:42.385895 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:42.385717    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-d6e26b\"/\"default-dockercfg-gl6dh\""
Sep 11 01:22:42.386090 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:42.386004    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-d6e26b\"/\"kube-root-ca.crt\""
Sep 11 01:22:42.392835 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:42.392809    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55"]
Sep 11 01:22:42.403240 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:42.403204    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:22:42.468108 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:42.468052    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-tmp-dir\") pod \"auth-test-model-d6e26b-kserve-865b575cbc-wjr55\" (UID: \"b6f32ac1-8184-42b3-b120-0d6e5c4fccc0\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55"
Sep 11 01:22:42.468108 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:42.468103    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-tls-certs\") pod \"auth-test-model-d6e26b-kserve-865b575cbc-wjr55\" (UID: \"b6f32ac1-8184-42b3-b120-0d6e5c4fccc0\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55"
Sep 11 01:22:42.468369 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:42.468126    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-model-cache\") pod \"auth-test-model-d6e26b-kserve-865b575cbc-wjr55\" (UID: \"b6f32ac1-8184-42b3-b120-0d6e5c4fccc0\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55"
Sep 11 01:22:42.468369 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:42.468224    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-home\") pod \"auth-test-model-d6e26b-kserve-865b575cbc-wjr55\" (UID: \"b6f32ac1-8184-42b3-b120-0d6e5c4fccc0\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55"
Sep 11 01:22:42.468369 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:42.468332    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-dshm\") pod \"auth-test-model-d6e26b-kserve-865b575cbc-wjr55\" (UID: \"b6f32ac1-8184-42b3-b120-0d6e5c4fccc0\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55"
Sep 11 01:22:42.468497 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:42.468394    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9c7v7\" (UniqueName: \"kubernetes.io/projected/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-kube-api-access-9c7v7\") pod \"auth-test-model-d6e26b-kserve-865b575cbc-wjr55\" (UID: \"b6f32ac1-8184-42b3-b120-0d6e5c4fccc0\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55"
Sep 11 01:22:42.569815 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:42.569772    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-tls-certs\") pod \"auth-test-model-d6e26b-kserve-865b575cbc-wjr55\" (UID: \"b6f32ac1-8184-42b3-b120-0d6e5c4fccc0\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55"
Sep 11 01:22:42.569815 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:42.569821    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-model-cache\") pod \"auth-test-model-d6e26b-kserve-865b575cbc-wjr55\" (UID: \"b6f32ac1-8184-42b3-b120-0d6e5c4fccc0\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55"
Sep 11 01:22:42.570129 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:42.569871    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-home\") pod \"auth-test-model-d6e26b-kserve-865b575cbc-wjr55\" (UID: \"b6f32ac1-8184-42b3-b120-0d6e5c4fccc0\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55"
Sep 11 01:22:42.570129 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:42.569980    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-dshm\") pod \"auth-test-model-d6e26b-kserve-865b575cbc-wjr55\" (UID: \"b6f32ac1-8184-42b3-b120-0d6e5c4fccc0\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55"
Sep 11 01:22:42.570129 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:42.570027    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9c7v7\" (UniqueName: \"kubernetes.io/projected/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-kube-api-access-9c7v7\") pod \"auth-test-model-d6e26b-kserve-865b575cbc-wjr55\" (UID: \"b6f32ac1-8184-42b3-b120-0d6e5c4fccc0\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55"
Sep 11 01:22:42.570129 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:42.570073    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-tmp-dir\") pod \"auth-test-model-d6e26b-kserve-865b575cbc-wjr55\" (UID: \"b6f32ac1-8184-42b3-b120-0d6e5c4fccc0\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55"
Sep 11 01:22:42.570377 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:42.570313    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-model-cache\") pod \"auth-test-model-d6e26b-kserve-865b575cbc-wjr55\" (UID: \"b6f32ac1-8184-42b3-b120-0d6e5c4fccc0\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55"
Sep 11 01:22:42.570377 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:42.570350    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-home\") pod \"auth-test-model-d6e26b-kserve-865b575cbc-wjr55\" (UID: \"b6f32ac1-8184-42b3-b120-0d6e5c4fccc0\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55"
Sep 11 01:22:42.570484 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:42.570383    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-tmp-dir\") pod \"auth-test-model-d6e26b-kserve-865b575cbc-wjr55\" (UID: \"b6f32ac1-8184-42b3-b120-0d6e5c4fccc0\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55"
Sep 11 01:22:42.572314 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:42.572294    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-dshm\") pod \"auth-test-model-d6e26b-kserve-865b575cbc-wjr55\" (UID: \"b6f32ac1-8184-42b3-b120-0d6e5c4fccc0\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55"
Sep 11 01:22:42.572467 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:42.572451    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-tls-certs\") pod \"auth-test-model-d6e26b-kserve-865b575cbc-wjr55\" (UID: \"b6f32ac1-8184-42b3-b120-0d6e5c4fccc0\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55"
Sep 11 01:22:42.579237 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:42.579212    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9c7v7\" (UniqueName: \"kubernetes.io/projected/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-kube-api-access-9c7v7\") pod \"auth-test-model-d6e26b-kserve-865b575cbc-wjr55\" (UID: \"b6f32ac1-8184-42b3-b120-0d6e5c4fccc0\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55"
Sep 11 01:22:42.693511 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:42.693474    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55"
Sep 11 01:22:42.987287 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:42.987211    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55"]
Sep 11 01:22:42.990722 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:22:42.990697    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb6f32ac1_8184_42b3_b120_0d6e5c4fccc0.slice/crio-a4b4f2b4efa3a52983d153df8f1b14c45a2a1ed5114623b195b16e438c76752b WatchSource:0}: Error finding container a4b4f2b4efa3a52983d153df8f1b14c45a2a1ed5114623b195b16e438c76752b: Status 404 returned error can't find the container with id a4b4f2b4efa3a52983d153df8f1b14c45a2a1ed5114623b195b16e438c76752b
Sep 11 01:22:43.332594 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:43.332494    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55" event={"ID":"b6f32ac1-8184-42b3-b120-0d6e5c4fccc0","Type":"ContainerStarted","Data":"49adcb220d11a1d3b29735a7621137fdaf1b416f131a2197382a1621db8f1737"}
Sep 11 01:22:43.332594 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:43.332540    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55" event={"ID":"b6f32ac1-8184-42b3-b120-0d6e5c4fccc0","Type":"ContainerStarted","Data":"a4b4f2b4efa3a52983d153df8f1b14c45a2a1ed5114623b195b16e438c76752b"}
Sep 11 01:22:43.353529 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:43.353472    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55" podStartSLOduration=1.353453077 podStartE2EDuration="1.353453077s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:22:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:22:43.351889259 +0000 UTC m=+1478.749081434" watchObservedRunningTime="2026-09-11 01:22:43.353453077 +0000 UTC m=+1478.750645250"
Sep 11 01:22:46.298618 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:46.298579    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:22:50.729334 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:50.729263    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6" podUID="fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc" containerName="main" containerID="cri-o://15b0990397ed15ae482e675f0948ead9fb7c39eb201b89a945d479efb61130e8" gracePeriod=45
Sep 11 01:22:50.985376 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:50.985317    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6"
Sep 11 01:22:51.051969 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:51.051939    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-home\") pod \"fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc\" (UID: \"fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc\") "
Sep 11 01:22:51.052125 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:51.051990    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-tls-certs\") pod \"fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc\" (UID: \"fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc\") "
Sep 11 01:22:51.052125 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:51.052020    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmkd8\" (UniqueName: \"kubernetes.io/projected/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-kube-api-access-bmkd8\") pod \"fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc\" (UID: \"fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc\") "
Sep 11 01:22:51.052125 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:51.052041    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-tmp-dir\") pod \"fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc\" (UID: \"fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc\") "
Sep 11 01:22:51.052301 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:51.052143    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-model-cache\") pod \"fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc\" (UID: \"fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc\") "
Sep 11 01:22:51.052301 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:51.052191    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-dshm\") pod \"fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc\" (UID: \"fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc\") "
Sep 11 01:22:51.052301 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:51.052233    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-home" (OuterVolumeSpecName: "home") pod "fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc" (UID: "fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:22:51.052525 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:51.052484    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-model-cache" (OuterVolumeSpecName: "model-cache") pod "fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc" (UID: "fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:22:51.052743 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:51.052606    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc" (UID: "fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:22:51.053057 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:51.053035    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-tmp-dir\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:22:51.053144 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:51.053063    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-model-cache\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:22:51.053144 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:51.053077    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-home\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:22:51.054654 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:51.054630    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-kube-api-access-bmkd8" (OuterVolumeSpecName: "kube-api-access-bmkd8") pod "fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc" (UID: "fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc"). InnerVolumeSpecName "kube-api-access-bmkd8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:22:51.054760 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:51.054700    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc" (UID: "fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:22:51.054878 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:51.054853    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-dshm" (OuterVolumeSpecName: "dshm") pod "fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc" (UID: "fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:22:51.154260 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:51.154227    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-tls-certs\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:22:51.154260 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:51.154252    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmkd8\" (UniqueName: \"kubernetes.io/projected/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-kube-api-access-bmkd8\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:22:51.154260 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:51.154261    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc-dshm\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:22:51.376873 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:51.376779    2574 generic.go:356] "Generic (PLEG): container finished" podID="fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc" containerID="15b0990397ed15ae482e675f0948ead9fb7c39eb201b89a945d479efb61130e8" exitCode=0
Sep 11 01:22:51.376873 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:51.376823    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6"
Sep 11 01:22:51.377138 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:51.376868    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6" event={"ID":"fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc","Type":"ContainerDied","Data":"15b0990397ed15ae482e675f0948ead9fb7c39eb201b89a945d479efb61130e8"}
Sep 11 01:22:51.377138 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:51.376938    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6" event={"ID":"fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc","Type":"ContainerDied","Data":"48e5b9b20558c4fa2fa2cca19e83bfefa7b23217a2b81bc162fa722da9d1ac39"}
Sep 11 01:22:51.377138 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:51.376959    2574 scope.go:117] "RemoveContainer" containerID="15b0990397ed15ae482e675f0948ead9fb7c39eb201b89a945d479efb61130e8"
Sep 11 01:22:51.387980 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:51.387939    2574 scope.go:117] "RemoveContainer" containerID="15b0990397ed15ae482e675f0948ead9fb7c39eb201b89a945d479efb61130e8"
Sep 11 01:22:51.388507 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:22:51.388477    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15b0990397ed15ae482e675f0948ead9fb7c39eb201b89a945d479efb61130e8\": container with ID starting with 15b0990397ed15ae482e675f0948ead9fb7c39eb201b89a945d479efb61130e8 not found: ID does not exist" containerID="15b0990397ed15ae482e675f0948ead9fb7c39eb201b89a945d479efb61130e8"
Sep 11 01:22:51.388566 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:51.388516    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15b0990397ed15ae482e675f0948ead9fb7c39eb201b89a945d479efb61130e8"} err="failed to get container status \"15b0990397ed15ae482e675f0948ead9fb7c39eb201b89a945d479efb61130e8\": rpc error: code = NotFound desc = could not find container \"15b0990397ed15ae482e675f0948ead9fb7c39eb201b89a945d479efb61130e8\": container with ID starting with 15b0990397ed15ae482e675f0948ead9fb7c39eb201b89a945d479efb61130e8 not found: ID does not exist"
Sep 11 01:22:51.391642 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:51.391619    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6"]
Sep 11 01:22:51.397336 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:51.397315    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-6f69f6e6/e2e-lifecycle-model-6f69f6e6-kserve-5d6465dfc5-mf8p6"]
Sep 11 01:22:52.693946 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:52.693900    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55"
Sep 11 01:22:52.694323 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:52.693963    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55"
Sep 11 01:22:52.706332 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:52.706310    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55"
Sep 11 01:22:53.100545 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:53.100449    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc" path="/var/lib/kubelet/pods/fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc/volumes"
Sep 11 01:22:53.399231 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:22:53.399149    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55"
Sep 11 01:23:04.145199 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:04.145166    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:23:11.849406 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:11.849372    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:23:12.618276 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:12.618240    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:23:24.203383 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:24.203339    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:23:26.957649 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:26.957574    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:23:27.024193 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:27.024148    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:23:27.238607 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:27.238527    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:23:28.661131 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:28.661097    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:23:28.752179 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:28.752152    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:23:28.854812 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:28.854785    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:23:29.529372 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:29.529338    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:23:29.626185 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:29.626153    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:23:29.724384 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:29.724341    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:23:29.827173 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:29.827110    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:23:31.013772 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:31.013729    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:23:32.837729 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:32.837693    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:23:37.033561 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:37.033534    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:23:38.290518 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:38.290486    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:23:38.565168 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:38.565094    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:23:38.677310 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:38.677278    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f4de10-p5c5s"]
Sep 11 01:23:38.677685 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:38.677672    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc" containerName="main"
Sep 11 01:23:38.677733 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:38.677688    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc" containerName="main"
Sep 11 01:23:38.677784 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:38.677774    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbf3e0de-16d4-4c8e-ab36-4d2ae019b3bc" containerName="main"
Sep 11 01:23:38.680952 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:38.680933    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f4de10-p5c5s"
Sep 11 01:23:38.683317 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:38.683299    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 01:23:38.683424 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:38.683334    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q6gbn\""
Sep 11 01:23:38.688412 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:38.688169    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f4de10-p5c5s"]
Sep 11 01:23:38.768927 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:38.768891    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99d4765d-2ce1-4c6d-b70d-9fa16b0e59c2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-f4de10-p5c5s\" (UID: \"99d4765d-2ce1-4c6d-b70d-9fa16b0e59c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f4de10-p5c5s"
Sep 11 01:23:38.848316 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:38.848219    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:23:38.869577 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:38.869554    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99d4765d-2ce1-4c6d-b70d-9fa16b0e59c2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-f4de10-p5c5s\" (UID: \"99d4765d-2ce1-4c6d-b70d-9fa16b0e59c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f4de10-p5c5s"
Sep 11 01:23:38.870146 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:38.870127    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99d4765d-2ce1-4c6d-b70d-9fa16b0e59c2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-f4de10-p5c5s\" (UID: \"99d4765d-2ce1-4c6d-b70d-9fa16b0e59c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f4de10-p5c5s"
Sep 11 01:23:38.992849 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:38.992827    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f4de10-p5c5s"
Sep 11 01:23:39.268391 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:39.268369    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f4de10-p5c5s"]
Sep 11 01:23:39.270207 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:23:39.270178    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99d4765d_2ce1_4c6d_b70d_9fa16b0e59c2.slice/crio-7c006231691f9015c516c44bc18e5168a4d669c1e36fdc7e352396b937de6e06 WatchSource:0}: Error finding container 7c006231691f9015c516c44bc18e5168a4d669c1e36fdc7e352396b937de6e06: Status 404 returned error can't find the container with id 7c006231691f9015c516c44bc18e5168a4d669c1e36fdc7e352396b937de6e06
Sep 11 01:23:39.632968 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:39.632938    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f4de10-p5c5s" event={"ID":"99d4765d-2ce1-4c6d-b70d-9fa16b0e59c2","Type":"ContainerStarted","Data":"8ad371b337d271db08173b17b09c2f47315b988a56546908ad55219b4677293a"}
Sep 11 01:23:39.633290 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:39.632986    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f4de10-p5c5s" event={"ID":"99d4765d-2ce1-4c6d-b70d-9fa16b0e59c2","Type":"ContainerStarted","Data":"7c006231691f9015c516c44bc18e5168a4d669c1e36fdc7e352396b937de6e06"}
Sep 11 01:23:39.647618 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:39.647573    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f4de10-p5c5s" podStartSLOduration=1.647556309 podStartE2EDuration="1.647556309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:23:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:23:39.647085854 +0000 UTC m=+1535.044278029" watchObservedRunningTime="2026-09-11 01:23:39.647556309 +0000 UTC m=+1535.044748483"
Sep 11 01:23:40.640025 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:40.639988    2574 generic.go:356] "Generic (PLEG): container finished" podID="99d4765d-2ce1-4c6d-b70d-9fa16b0e59c2" containerID="8ad371b337d271db08173b17b09c2f47315b988a56546908ad55219b4677293a" exitCode=0
Sep 11 01:23:40.640438 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:40.640113    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f4de10-p5c5s" event={"ID":"99d4765d-2ce1-4c6d-b70d-9fa16b0e59c2","Type":"ContainerDied","Data":"8ad371b337d271db08173b17b09c2f47315b988a56546908ad55219b4677293a"}
Sep 11 01:23:41.779037 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:41.779014    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f4de10-p5c5s"
Sep 11 01:23:41.897051 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:41.896985    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99d4765d-2ce1-4c6d-b70d-9fa16b0e59c2-maas-api-service-ca\") pod \"99d4765d-2ce1-4c6d-b70d-9fa16b0e59c2\" (UID: \"99d4765d-2ce1-4c6d-b70d-9fa16b0e59c2\") "
Sep 11 01:23:41.897312 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:41.897289    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99d4765d-2ce1-4c6d-b70d-9fa16b0e59c2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "99d4765d-2ce1-4c6d-b70d-9fa16b0e59c2" (UID: "99d4765d-2ce1-4c6d-b70d-9fa16b0e59c2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:23:41.997752 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:41.997726    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99d4765d-2ce1-4c6d-b70d-9fa16b0e59c2-maas-api-service-ca\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:23:42.652127 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:42.652097    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f4de10-p5c5s"
Sep 11 01:23:42.652303 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:42.652102    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f4de10-p5c5s" event={"ID":"99d4765d-2ce1-4c6d-b70d-9fa16b0e59c2","Type":"ContainerDied","Data":"7c006231691f9015c516c44bc18e5168a4d669c1e36fdc7e352396b937de6e06"}
Sep 11 01:23:42.652303 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:42.652218    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7c006231691f9015c516c44bc18e5168a4d669c1e36fdc7e352396b937de6e06"
Sep 11 01:23:43.428838 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:43.428803    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:23:43.488467 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:43.488434    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:23:43.650438 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:43.650404    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:23:43.650747 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:43.650726    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8"]
Sep 11 01:23:43.661110 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:43.661081    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt"]
Sep 11 01:23:43.675696 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:43.675673    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt"]
Sep 11 01:23:43.731820 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:43.731751    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:23:43.798768 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:43.798742    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:23:49.277458 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:49.277427    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:23:54.245700 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:54.245665    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:23:54.550073 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:54.549984    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:23:54.960899 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:54.960868    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:23:54.968520 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:54.968478    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f4de10-29818155-7n5jw"]
Sep 11 01:23:54.970316 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:54.970296    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f4de10-29818155-7n5jw"]
Sep 11 01:23:54.992598 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:54.992574    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-f4de10-857b967875-n67cq"]
Sep 11 01:23:54.992807 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:54.992782    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-f4de10-857b967875-n67cq" podUID="c4337d7d-b9d3-43f4-9cb8-5ae74419c273" containerName="payload-processing" containerID="cri-o://8021eb8446c812146a5e4314da5efe5a3db20f5e556ba12e3d715e23fec294d5" gracePeriod=30
Sep 11 01:23:54.998703 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:54.998679    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-f4de10-977ff8d6-6g49f"]
Sep 11 01:23:54.999085 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:54.999056    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f4de10-977ff8d6-6g49f" podUID="c67d64bb-40f3-4540-a3a0-8001a339a21c" containerName="payload-pre-processing" containerID="cri-o://7dd2ba2d1c951873b3830284a4c85b2642b851f3b5e58a28e7a9ebcfbd521802" gracePeriod=30
Sep 11 01:23:55.030524 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.030497    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:23:55.097678 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.097641    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:23:55.100550 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.100525    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a4144ca-0da8-4af7-b7d4-076d8bf18d8e" path="/var/lib/kubelet/pods/6a4144ca-0da8-4af7-b7d4-076d8bf18d8e/volumes"
Sep 11 01:23:55.175252 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.175220    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f4de10-p5c5s"]
Sep 11 01:23:55.180494 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.180468    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:23:55.181689 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.181644    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f4de10-p5c5s"]
Sep 11 01:23:55.242932 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.242878    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:23:55.250158 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.250135    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-f4de10-857b967875-n67cq"
Sep 11 01:23:55.275578 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.275558    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f4de10-977ff8d6-6g49f"
Sep 11 01:23:55.310511 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.310485    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jv4db\" (UniqueName: \"kubernetes.io/projected/c4337d7d-b9d3-43f4-9cb8-5ae74419c273-kube-api-access-jv4db\") pod \"c4337d7d-b9d3-43f4-9cb8-5ae74419c273\" (UID: \"c4337d7d-b9d3-43f4-9cb8-5ae74419c273\") "
Sep 11 01:23:55.310631 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.310549    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4337d7d-b9d3-43f4-9cb8-5ae74419c273-plugins-config-volume\") pod \"c4337d7d-b9d3-43f4-9cb8-5ae74419c273\" (UID: \"c4337d7d-b9d3-43f4-9cb8-5ae74419c273\") "
Sep 11 01:23:55.310927 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.310887    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4337d7d-b9d3-43f4-9cb8-5ae74419c273-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c4337d7d-b9d3-43f4-9cb8-5ae74419c273" (UID: "c4337d7d-b9d3-43f4-9cb8-5ae74419c273"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:23:55.312682 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.312663    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4337d7d-b9d3-43f4-9cb8-5ae74419c273-kube-api-access-jv4db" (OuterVolumeSpecName: "kube-api-access-jv4db") pod "c4337d7d-b9d3-43f4-9cb8-5ae74419c273" (UID: "c4337d7d-b9d3-43f4-9cb8-5ae74419c273"). InnerVolumeSpecName "kube-api-access-jv4db". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:23:55.411391 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.411367    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c67d64bb-40f3-4540-a3a0-8001a339a21c-plugins-config-volume\") pod \"c67d64bb-40f3-4540-a3a0-8001a339a21c\" (UID: \"c67d64bb-40f3-4540-a3a0-8001a339a21c\") "
Sep 11 01:23:55.411499 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.411407    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ml6c7\" (UniqueName: \"kubernetes.io/projected/c67d64bb-40f3-4540-a3a0-8001a339a21c-kube-api-access-ml6c7\") pod \"c67d64bb-40f3-4540-a3a0-8001a339a21c\" (UID: \"c67d64bb-40f3-4540-a3a0-8001a339a21c\") "
Sep 11 01:23:55.411742 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.411721    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jv4db\" (UniqueName: \"kubernetes.io/projected/c4337d7d-b9d3-43f4-9cb8-5ae74419c273-kube-api-access-jv4db\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:23:55.411742 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.411733    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c67d64bb-40f3-4540-a3a0-8001a339a21c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c67d64bb-40f3-4540-a3a0-8001a339a21c" (UID: "c67d64bb-40f3-4540-a3a0-8001a339a21c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:23:55.411841 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.411748    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4337d7d-b9d3-43f4-9cb8-5ae74419c273-plugins-config-volume\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:23:55.413479 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.413456    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c67d64bb-40f3-4540-a3a0-8001a339a21c-kube-api-access-ml6c7" (OuterVolumeSpecName: "kube-api-access-ml6c7") pod "c67d64bb-40f3-4540-a3a0-8001a339a21c" (UID: "c67d64bb-40f3-4540-a3a0-8001a339a21c"). InnerVolumeSpecName "kube-api-access-ml6c7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:23:55.512392 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.512330    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c67d64bb-40f3-4540-a3a0-8001a339a21c-plugins-config-volume\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:23:55.512392 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.512352    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ml6c7\" (UniqueName: \"kubernetes.io/projected/c67d64bb-40f3-4540-a3a0-8001a339a21c-kube-api-access-ml6c7\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:23:55.653985 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.653955    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:23:55.720835 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.720800    2574 generic.go:356] "Generic (PLEG): container finished" podID="c4337d7d-b9d3-43f4-9cb8-5ae74419c273" containerID="8021eb8446c812146a5e4314da5efe5a3db20f5e556ba12e3d715e23fec294d5" exitCode=0
Sep 11 01:23:55.721005 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.720844    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-f4de10-857b967875-n67cq"
Sep 11 01:23:55.721005 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.720886    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-f4de10-857b967875-n67cq" event={"ID":"c4337d7d-b9d3-43f4-9cb8-5ae74419c273","Type":"ContainerDied","Data":"8021eb8446c812146a5e4314da5efe5a3db20f5e556ba12e3d715e23fec294d5"}
Sep 11 01:23:55.721005 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.720962    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-f4de10-857b967875-n67cq" event={"ID":"c4337d7d-b9d3-43f4-9cb8-5ae74419c273","Type":"ContainerDied","Data":"718e9a230a1701a4105c9af769d5646ab256b14c10655aa07217c3114b718c8d"}
Sep 11 01:23:55.721005 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.720990    2574 scope.go:117] "RemoveContainer" containerID="8021eb8446c812146a5e4314da5efe5a3db20f5e556ba12e3d715e23fec294d5"
Sep 11 01:23:55.722357 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.722335    2574 generic.go:356] "Generic (PLEG): container finished" podID="c67d64bb-40f3-4540-a3a0-8001a339a21c" containerID="7dd2ba2d1c951873b3830284a4c85b2642b851f3b5e58a28e7a9ebcfbd521802" exitCode=0
Sep 11 01:23:55.722472 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.722375    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f4de10-977ff8d6-6g49f" event={"ID":"c67d64bb-40f3-4540-a3a0-8001a339a21c","Type":"ContainerDied","Data":"7dd2ba2d1c951873b3830284a4c85b2642b851f3b5e58a28e7a9ebcfbd521802"}
Sep 11 01:23:55.722472 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.722396    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f4de10-977ff8d6-6g49f"
Sep 11 01:23:55.722472 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.722440    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-f4de10-977ff8d6-6g49f" event={"ID":"c67d64bb-40f3-4540-a3a0-8001a339a21c","Type":"ContainerDied","Data":"990ce1d1d3a3fb6fd100a13e672529cb34828673667b08255de28530381e1373"}
Sep 11 01:23:55.731744 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.731725    2574 scope.go:117] "RemoveContainer" containerID="8021eb8446c812146a5e4314da5efe5a3db20f5e556ba12e3d715e23fec294d5"
Sep 11 01:23:55.731996 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:23:55.731978    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8021eb8446c812146a5e4314da5efe5a3db20f5e556ba12e3d715e23fec294d5\": container with ID starting with 8021eb8446c812146a5e4314da5efe5a3db20f5e556ba12e3d715e23fec294d5 not found: ID does not exist" containerID="8021eb8446c812146a5e4314da5efe5a3db20f5e556ba12e3d715e23fec294d5"
Sep 11 01:23:55.732069 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.732003    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8021eb8446c812146a5e4314da5efe5a3db20f5e556ba12e3d715e23fec294d5"} err="failed to get container status \"8021eb8446c812146a5e4314da5efe5a3db20f5e556ba12e3d715e23fec294d5\": rpc error: code = NotFound desc = could not find container \"8021eb8446c812146a5e4314da5efe5a3db20f5e556ba12e3d715e23fec294d5\": container with ID starting with 8021eb8446c812146a5e4314da5efe5a3db20f5e556ba12e3d715e23fec294d5 not found: ID does not exist"
Sep 11 01:23:55.732069 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.732019    2574 scope.go:117] "RemoveContainer" containerID="7dd2ba2d1c951873b3830284a4c85b2642b851f3b5e58a28e7a9ebcfbd521802"
Sep 11 01:23:55.739810 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.739787    2574 scope.go:117] "RemoveContainer" containerID="7dd2ba2d1c951873b3830284a4c85b2642b851f3b5e58a28e7a9ebcfbd521802"
Sep 11 01:23:55.740076 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:23:55.740060    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7dd2ba2d1c951873b3830284a4c85b2642b851f3b5e58a28e7a9ebcfbd521802\": container with ID starting with 7dd2ba2d1c951873b3830284a4c85b2642b851f3b5e58a28e7a9ebcfbd521802 not found: ID does not exist" containerID="7dd2ba2d1c951873b3830284a4c85b2642b851f3b5e58a28e7a9ebcfbd521802"
Sep 11 01:23:55.740138 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.740081    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dd2ba2d1c951873b3830284a4c85b2642b851f3b5e58a28e7a9ebcfbd521802"} err="failed to get container status \"7dd2ba2d1c951873b3830284a4c85b2642b851f3b5e58a28e7a9ebcfbd521802\": rpc error: code = NotFound desc = could not find container \"7dd2ba2d1c951873b3830284a4c85b2642b851f3b5e58a28e7a9ebcfbd521802\": container with ID starting with 7dd2ba2d1c951873b3830284a4c85b2642b851f3b5e58a28e7a9ebcfbd521802 not found: ID does not exist"
Sep 11 01:23:55.743587 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.743565    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-f4de10-857b967875-n67cq"]
Sep 11 01:23:55.747333 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.747313    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-f4de10-857b967875-n67cq"]
Sep 11 01:23:55.756652 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.756632    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-f4de10-977ff8d6-6g49f"]
Sep 11 01:23:55.760132 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:55.760113    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-f4de10-977ff8d6-6g49f"]
Sep 11 01:23:57.099804 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:57.099772    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99d4765d-2ce1-4c6d-b70d-9fa16b0e59c2" path="/var/lib/kubelet/pods/99d4765d-2ce1-4c6d-b70d-9fa16b0e59c2/volumes"
Sep 11 01:23:57.100194 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:57.100140    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4337d7d-b9d3-43f4-9cb8-5ae74419c273" path="/var/lib/kubelet/pods/c4337d7d-b9d3-43f4-9cb8-5ae74419c273/volumes"
Sep 11 01:23:57.100448 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:57.100435    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c67d64bb-40f3-4540-a3a0-8001a339a21c" path="/var/lib/kubelet/pods/c67d64bb-40f3-4540-a3a0-8001a339a21c/volumes"
Sep 11 01:23:58.671199 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:58.671156    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8" podUID="7159c6eb-0eac-492d-a5a0-720156e16245" containerName="main" containerID="cri-o://e0e9e0bcc9764ae655a745924cbff741c6ede6873ac7014e1ef9481d6607d554" gracePeriod=45
Sep 11 01:23:58.685390 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:58.685271    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt" podUID="3e8b3e1d-7599-4c9f-bea1-e921bbb11d91" containerName="main" containerID="cri-o://211a95460b552196bd87a31d0ed6801ddea4601b1f9aa5a697f2f2d0d2bc60d1" gracePeriod=45
Sep 11 01:23:58.694547 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:58.694514    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt" podUID="f75a0b21-b351-4447-8774-e7b34f3fd258" containerName="main" containerID="cri-o://46482d198d8ead12b7d6f94720896cd8788c557d556f5b527287349783435dce" gracePeriod=45
Sep 11 01:23:58.962278 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:58.962252    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt"
Sep 11 01:23:58.985061 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:58.984793    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8"
Sep 11 01:23:58.989870 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:58.989845    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt"
Sep 11 01:23:59.039849 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.039805    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-tls-certs\") pod \"3e8b3e1d-7599-4c9f-bea1-e921bbb11d91\" (UID: \"3e8b3e1d-7599-4c9f-bea1-e921bbb11d91\") "
Sep 11 01:23:59.040032 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.039859    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7159c6eb-0eac-492d-a5a0-720156e16245-tmp-dir\") pod \"7159c6eb-0eac-492d-a5a0-720156e16245\" (UID: \"7159c6eb-0eac-492d-a5a0-720156e16245\") "
Sep 11 01:23:59.040032 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.039896    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-home\") pod \"3e8b3e1d-7599-4c9f-bea1-e921bbb11d91\" (UID: \"3e8b3e1d-7599-4c9f-bea1-e921bbb11d91\") "
Sep 11 01:23:59.040032 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.039954    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7159c6eb-0eac-492d-a5a0-720156e16245-model-cache\") pod \"7159c6eb-0eac-492d-a5a0-720156e16245\" (UID: \"7159c6eb-0eac-492d-a5a0-720156e16245\") "
Sep 11 01:23:59.040032 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.039980    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-tmp-dir\") pod \"3e8b3e1d-7599-4c9f-bea1-e921bbb11d91\" (UID: \"3e8b3e1d-7599-4c9f-bea1-e921bbb11d91\") "
Sep 11 01:23:59.040250 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.040053    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5dvhk\" (UniqueName: \"kubernetes.io/projected/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-kube-api-access-5dvhk\") pod \"3e8b3e1d-7599-4c9f-bea1-e921bbb11d91\" (UID: \"3e8b3e1d-7599-4c9f-bea1-e921bbb11d91\") "
Sep 11 01:23:59.040250 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.040087    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7159c6eb-0eac-492d-a5a0-720156e16245-tls-certs\") pod \"7159c6eb-0eac-492d-a5a0-720156e16245\" (UID: \"7159c6eb-0eac-492d-a5a0-720156e16245\") "
Sep 11 01:23:59.040250 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.040161    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7h9g\" (UniqueName: \"kubernetes.io/projected/7159c6eb-0eac-492d-a5a0-720156e16245-kube-api-access-x7h9g\") pod \"7159c6eb-0eac-492d-a5a0-720156e16245\" (UID: \"7159c6eb-0eac-492d-a5a0-720156e16245\") "
Sep 11 01:23:59.040250 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.040197    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7159c6eb-0eac-492d-a5a0-720156e16245-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7159c6eb-0eac-492d-a5a0-720156e16245" (UID: "7159c6eb-0eac-492d-a5a0-720156e16245"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:23:59.040250 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.040199    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-home" (OuterVolumeSpecName: "home") pod "3e8b3e1d-7599-4c9f-bea1-e921bbb11d91" (UID: "3e8b3e1d-7599-4c9f-bea1-e921bbb11d91"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:23:59.040250 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.040228    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7159c6eb-0eac-492d-a5a0-720156e16245-dshm\") pod \"7159c6eb-0eac-492d-a5a0-720156e16245\" (UID: \"7159c6eb-0eac-492d-a5a0-720156e16245\") "
Sep 11 01:23:59.040551 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.040292    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-dshm\") pod \"3e8b3e1d-7599-4c9f-bea1-e921bbb11d91\" (UID: \"3e8b3e1d-7599-4c9f-bea1-e921bbb11d91\") "
Sep 11 01:23:59.040551 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.040326    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7159c6eb-0eac-492d-a5a0-720156e16245-home\") pod \"7159c6eb-0eac-492d-a5a0-720156e16245\" (UID: \"7159c6eb-0eac-492d-a5a0-720156e16245\") "
Sep 11 01:23:59.040551 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.040406    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-model-cache\") pod \"3e8b3e1d-7599-4c9f-bea1-e921bbb11d91\" (UID: \"3e8b3e1d-7599-4c9f-bea1-e921bbb11d91\") "
Sep 11 01:23:59.040876 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.040754    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7159c6eb-0eac-492d-a5a0-720156e16245-tmp-dir\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:23:59.040876 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.040775    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-home\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:23:59.041014 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.040967    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3e8b3e1d-7599-4c9f-bea1-e921bbb11d91" (UID: "3e8b3e1d-7599-4c9f-bea1-e921bbb11d91"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:23:59.041014 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.040984    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-model-cache" (OuterVolumeSpecName: "model-cache") pod "3e8b3e1d-7599-4c9f-bea1-e921bbb11d91" (UID: "3e8b3e1d-7599-4c9f-bea1-e921bbb11d91"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:23:59.041374 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.041199    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7159c6eb-0eac-492d-a5a0-720156e16245-model-cache" (OuterVolumeSpecName: "model-cache") pod "7159c6eb-0eac-492d-a5a0-720156e16245" (UID: "7159c6eb-0eac-492d-a5a0-720156e16245"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:23:59.041722 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.041701    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7159c6eb-0eac-492d-a5a0-720156e16245-home" (OuterVolumeSpecName: "home") pod "7159c6eb-0eac-492d-a5a0-720156e16245" (UID: "7159c6eb-0eac-492d-a5a0-720156e16245"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:23:59.043235 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.043188    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7159c6eb-0eac-492d-a5a0-720156e16245-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7159c6eb-0eac-492d-a5a0-720156e16245" (UID: "7159c6eb-0eac-492d-a5a0-720156e16245"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:23:59.043235 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.043202    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3e8b3e1d-7599-4c9f-bea1-e921bbb11d91" (UID: "3e8b3e1d-7599-4c9f-bea1-e921bbb11d91"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:23:59.043235 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.043225    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7159c6eb-0eac-492d-a5a0-720156e16245-dshm" (OuterVolumeSpecName: "dshm") pod "7159c6eb-0eac-492d-a5a0-720156e16245" (UID: "7159c6eb-0eac-492d-a5a0-720156e16245"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:23:59.043549 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.043520    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7159c6eb-0eac-492d-a5a0-720156e16245-kube-api-access-x7h9g" (OuterVolumeSpecName: "kube-api-access-x7h9g") pod "7159c6eb-0eac-492d-a5a0-720156e16245" (UID: "7159c6eb-0eac-492d-a5a0-720156e16245"). InnerVolumeSpecName "kube-api-access-x7h9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:23:59.043877 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.043854    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-kube-api-access-5dvhk" (OuterVolumeSpecName: "kube-api-access-5dvhk") pod "3e8b3e1d-7599-4c9f-bea1-e921bbb11d91" (UID: "3e8b3e1d-7599-4c9f-bea1-e921bbb11d91"). InnerVolumeSpecName "kube-api-access-5dvhk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:23:59.043976 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.043959    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-dshm" (OuterVolumeSpecName: "dshm") pod "3e8b3e1d-7599-4c9f-bea1-e921bbb11d91" (UID: "3e8b3e1d-7599-4c9f-bea1-e921bbb11d91"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:23:59.141212 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.141181    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f75a0b21-b351-4447-8774-e7b34f3fd258-model-cache\") pod \"f75a0b21-b351-4447-8774-e7b34f3fd258\" (UID: \"f75a0b21-b351-4447-8774-e7b34f3fd258\") "
Sep 11 01:23:59.141321 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.141241    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f75a0b21-b351-4447-8774-e7b34f3fd258-home\") pod \"f75a0b21-b351-4447-8774-e7b34f3fd258\" (UID: \"f75a0b21-b351-4447-8774-e7b34f3fd258\") "
Sep 11 01:23:59.141321 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.141271    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f75a0b21-b351-4447-8774-e7b34f3fd258-tmp-dir\") pod \"f75a0b21-b351-4447-8774-e7b34f3fd258\" (UID: \"f75a0b21-b351-4447-8774-e7b34f3fd258\") "
Sep 11 01:23:59.141321 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.141307    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f75a0b21-b351-4447-8774-e7b34f3fd258-tls-certs\") pod \"f75a0b21-b351-4447-8774-e7b34f3fd258\" (UID: \"f75a0b21-b351-4447-8774-e7b34f3fd258\") "
Sep 11 01:23:59.141486 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.141358    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f75a0b21-b351-4447-8774-e7b34f3fd258-dshm\") pod \"f75a0b21-b351-4447-8774-e7b34f3fd258\" (UID: \"f75a0b21-b351-4447-8774-e7b34f3fd258\") "
Sep 11 01:23:59.141486 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.141414    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zd4nd\" (UniqueName: \"kubernetes.io/projected/f75a0b21-b351-4447-8774-e7b34f3fd258-kube-api-access-zd4nd\") pod \"f75a0b21-b351-4447-8774-e7b34f3fd258\" (UID: \"f75a0b21-b351-4447-8774-e7b34f3fd258\") "
Sep 11 01:23:59.141486 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.141415    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f75a0b21-b351-4447-8774-e7b34f3fd258-model-cache" (OuterVolumeSpecName: "model-cache") pod "f75a0b21-b351-4447-8774-e7b34f3fd258" (UID: "f75a0b21-b351-4447-8774-e7b34f3fd258"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:23:59.141637 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.141528    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f75a0b21-b351-4447-8774-e7b34f3fd258-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f75a0b21-b351-4447-8774-e7b34f3fd258" (UID: "f75a0b21-b351-4447-8774-e7b34f3fd258"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:23:59.141637 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.141538    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f75a0b21-b351-4447-8774-e7b34f3fd258-home" (OuterVolumeSpecName: "home") pod "f75a0b21-b351-4447-8774-e7b34f3fd258" (UID: "f75a0b21-b351-4447-8774-e7b34f3fd258"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:23:59.141738 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.141698    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7159c6eb-0eac-492d-a5a0-720156e16245-model-cache\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:23:59.141738 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.141717    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-tmp-dir\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:23:59.141738 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.141730    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f75a0b21-b351-4447-8774-e7b34f3fd258-model-cache\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:23:59.141951 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.141744    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dvhk\" (UniqueName: \"kubernetes.io/projected/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-kube-api-access-5dvhk\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:23:59.141951 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.141757    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7159c6eb-0eac-492d-a5a0-720156e16245-tls-certs\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:23:59.141951 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.141770    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f75a0b21-b351-4447-8774-e7b34f3fd258-home\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:23:59.141951 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.141781    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f75a0b21-b351-4447-8774-e7b34f3fd258-tmp-dir\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:23:59.141951 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.141793    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7h9g\" (UniqueName: \"kubernetes.io/projected/7159c6eb-0eac-492d-a5a0-720156e16245-kube-api-access-x7h9g\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:23:59.141951 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.141805    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7159c6eb-0eac-492d-a5a0-720156e16245-dshm\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:23:59.141951 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.141818    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-dshm\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:23:59.141951 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.141829    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7159c6eb-0eac-492d-a5a0-720156e16245-home\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:23:59.141951 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.141841    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-model-cache\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:23:59.141951 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.141853    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91-tls-certs\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:23:59.143605 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.143585    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f75a0b21-b351-4447-8774-e7b34f3fd258-dshm" (OuterVolumeSpecName: "dshm") pod "f75a0b21-b351-4447-8774-e7b34f3fd258" (UID: "f75a0b21-b351-4447-8774-e7b34f3fd258"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:23:59.143669 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.143614    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f75a0b21-b351-4447-8774-e7b34f3fd258-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f75a0b21-b351-4447-8774-e7b34f3fd258" (UID: "f75a0b21-b351-4447-8774-e7b34f3fd258"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:23:59.143669 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.143660    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f75a0b21-b351-4447-8774-e7b34f3fd258-kube-api-access-zd4nd" (OuterVolumeSpecName: "kube-api-access-zd4nd") pod "f75a0b21-b351-4447-8774-e7b34f3fd258" (UID: "f75a0b21-b351-4447-8774-e7b34f3fd258"). InnerVolumeSpecName "kube-api-access-zd4nd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:23:59.243293 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.243233    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f75a0b21-b351-4447-8774-e7b34f3fd258-dshm\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:23:59.243293 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.243257    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zd4nd\" (UniqueName: \"kubernetes.io/projected/f75a0b21-b351-4447-8774-e7b34f3fd258-kube-api-access-zd4nd\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:23:59.243293 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.243279    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f75a0b21-b351-4447-8774-e7b34f3fd258-tls-certs\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:23:59.748274 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.747902    2574 generic.go:356] "Generic (PLEG): container finished" podID="7159c6eb-0eac-492d-a5a0-720156e16245" containerID="e0e9e0bcc9764ae655a745924cbff741c6ede6873ac7014e1ef9481d6607d554" exitCode=0
Sep 11 01:23:59.748274 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.748027    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8"
Sep 11 01:23:59.748274 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.748086    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8" event={"ID":"7159c6eb-0eac-492d-a5a0-720156e16245","Type":"ContainerDied","Data":"e0e9e0bcc9764ae655a745924cbff741c6ede6873ac7014e1ef9481d6607d554"}
Sep 11 01:23:59.748274 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.748123    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8" event={"ID":"7159c6eb-0eac-492d-a5a0-720156e16245","Type":"ContainerDied","Data":"cf02cfbb98d9ee56484d43d36b6b9d240d2d7bc083c8945f7dd3f914a230123f"}
Sep 11 01:23:59.748274 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.748149    2574 scope.go:117] "RemoveContainer" containerID="e0e9e0bcc9764ae655a745924cbff741c6ede6873ac7014e1ef9481d6607d554"
Sep 11 01:23:59.750303 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.750199    2574 generic.go:356] "Generic (PLEG): container finished" podID="f75a0b21-b351-4447-8774-e7b34f3fd258" containerID="46482d198d8ead12b7d6f94720896cd8788c557d556f5b527287349783435dce" exitCode=0
Sep 11 01:23:59.750303 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.750257    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt"
Sep 11 01:23:59.750424 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.750324    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt" event={"ID":"f75a0b21-b351-4447-8774-e7b34f3fd258","Type":"ContainerDied","Data":"46482d198d8ead12b7d6f94720896cd8788c557d556f5b527287349783435dce"}
Sep 11 01:23:59.750424 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.750358    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt" event={"ID":"f75a0b21-b351-4447-8774-e7b34f3fd258","Type":"ContainerDied","Data":"77851c1b50703a876200a4961af839d0c4191ffd2d430baf6abab7061200fbb2"}
Sep 11 01:23:59.752051 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.752029    2574 generic.go:356] "Generic (PLEG): container finished" podID="3e8b3e1d-7599-4c9f-bea1-e921bbb11d91" containerID="211a95460b552196bd87a31d0ed6801ddea4601b1f9aa5a697f2f2d0d2bc60d1" exitCode=0
Sep 11 01:23:59.752167 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.752062    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt"
Sep 11 01:23:59.752167 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.752066    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt" event={"ID":"3e8b3e1d-7599-4c9f-bea1-e921bbb11d91","Type":"ContainerDied","Data":"211a95460b552196bd87a31d0ed6801ddea4601b1f9aa5a697f2f2d0d2bc60d1"}
Sep 11 01:23:59.752167 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.752094    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt" event={"ID":"3e8b3e1d-7599-4c9f-bea1-e921bbb11d91","Type":"ContainerDied","Data":"85b16bc62f4d5756e85ba71ac371db20ffc5e91a8f5979ee15f6251dbe1fecea"}
Sep 11 01:23:59.758482 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.758457    2574 scope.go:117] "RemoveContainer" containerID="e0e9e0bcc9764ae655a745924cbff741c6ede6873ac7014e1ef9481d6607d554"
Sep 11 01:23:59.758794 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:23:59.758764    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0e9e0bcc9764ae655a745924cbff741c6ede6873ac7014e1ef9481d6607d554\": container with ID starting with e0e9e0bcc9764ae655a745924cbff741c6ede6873ac7014e1ef9481d6607d554 not found: ID does not exist" containerID="e0e9e0bcc9764ae655a745924cbff741c6ede6873ac7014e1ef9481d6607d554"
Sep 11 01:23:59.758941 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.758805    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0e9e0bcc9764ae655a745924cbff741c6ede6873ac7014e1ef9481d6607d554"} err="failed to get container status \"e0e9e0bcc9764ae655a745924cbff741c6ede6873ac7014e1ef9481d6607d554\": rpc error: code = NotFound desc = could not find container \"e0e9e0bcc9764ae655a745924cbff741c6ede6873ac7014e1ef9481d6607d554\": container with ID starting with e0e9e0bcc9764ae655a745924cbff741c6ede6873ac7014e1ef9481d6607d554 not found: ID does not exist"
Sep 11 01:23:59.758941 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.758828    2574 scope.go:117] "RemoveContainer" containerID="46482d198d8ead12b7d6f94720896cd8788c557d556f5b527287349783435dce"
Sep 11 01:23:59.764873 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.764851    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8"]
Sep 11 01:23:59.767171 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.767157    2574 scope.go:117] "RemoveContainer" containerID="46482d198d8ead12b7d6f94720896cd8788c557d556f5b527287349783435dce"
Sep 11 01:23:59.767413 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:23:59.767397    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46482d198d8ead12b7d6f94720896cd8788c557d556f5b527287349783435dce\": container with ID starting with 46482d198d8ead12b7d6f94720896cd8788c557d556f5b527287349783435dce not found: ID does not exist" containerID="46482d198d8ead12b7d6f94720896cd8788c557d556f5b527287349783435dce"
Sep 11 01:23:59.767465 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.767419    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46482d198d8ead12b7d6f94720896cd8788c557d556f5b527287349783435dce"} err="failed to get container status \"46482d198d8ead12b7d6f94720896cd8788c557d556f5b527287349783435dce\": rpc error: code = NotFound desc = could not find container \"46482d198d8ead12b7d6f94720896cd8788c557d556f5b527287349783435dce\": container with ID starting with 46482d198d8ead12b7d6f94720896cd8788c557d556f5b527287349783435dce not found: ID does not exist"
Sep 11 01:23:59.767465 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.767432    2574 scope.go:117] "RemoveContainer" containerID="211a95460b552196bd87a31d0ed6801ddea4601b1f9aa5a697f2f2d0d2bc60d1"
Sep 11 01:23:59.772190 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.772171    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-f4de10/distinct-2-w0-f4de10-kserve-5b69c6bc9b-crls8"]
Sep 11 01:23:59.775930 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.775899    2574 scope.go:117] "RemoveContainer" containerID="211a95460b552196bd87a31d0ed6801ddea4601b1f9aa5a697f2f2d0d2bc60d1"
Sep 11 01:23:59.776184 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:23:59.776167    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"211a95460b552196bd87a31d0ed6801ddea4601b1f9aa5a697f2f2d0d2bc60d1\": container with ID starting with 211a95460b552196bd87a31d0ed6801ddea4601b1f9aa5a697f2f2d0d2bc60d1 not found: ID does not exist" containerID="211a95460b552196bd87a31d0ed6801ddea4601b1f9aa5a697f2f2d0d2bc60d1"
Sep 11 01:23:59.776230 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.776200    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"211a95460b552196bd87a31d0ed6801ddea4601b1f9aa5a697f2f2d0d2bc60d1"} err="failed to get container status \"211a95460b552196bd87a31d0ed6801ddea4601b1f9aa5a697f2f2d0d2bc60d1\": rpc error: code = NotFound desc = could not find container \"211a95460b552196bd87a31d0ed6801ddea4601b1f9aa5a697f2f2d0d2bc60d1\": container with ID starting with 211a95460b552196bd87a31d0ed6801ddea4601b1f9aa5a697f2f2d0d2bc60d1 not found: ID does not exist"
Sep 11 01:23:59.785706 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.785685    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt"]
Sep 11 01:23:59.789624 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.789603    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-f4de10/premium-w0-f4de10-kserve-7787487fb8-cswlt"]
Sep 11 01:23:59.798696 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.798678    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt"]
Sep 11 01:23:59.802289 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:23:59.802270    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-f4de10/embedding-w0-f4de10-kserve-d4f94cb44-kztkt"]
Sep 11 01:24:01.100192 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:01.100162    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e8b3e1d-7599-4c9f-bea1-e921bbb11d91" path="/var/lib/kubelet/pods/3e8b3e1d-7599-4c9f-bea1-e921bbb11d91/volumes"
Sep 11 01:24:01.100556 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:01.100538    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7159c6eb-0eac-492d-a5a0-720156e16245" path="/var/lib/kubelet/pods/7159c6eb-0eac-492d-a5a0-720156e16245/volumes"
Sep 11 01:24:01.100876 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:01.100863    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f75a0b21-b351-4447-8774-e7b34f3fd258" path="/var/lib/kubelet/pods/f75a0b21-b351-4447-8774-e7b34f3fd258/volumes"
Sep 11 01:24:01.466081 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:01.466042    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:24:01.527292 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:01.527259    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:24:01.736465 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:01.736394    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:24:01.986315 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:01.986271    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:24:02.086839 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:02.086759    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:24:05.436525 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:05.436497    2574 scope.go:117] "RemoveContainer" containerID="c98534f7ffb57dea64748b958843202510c80fe104379e82d8e2bc2650e3497b"
Sep 11 01:24:05.445797 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:05.445745    2574 scope.go:117] "RemoveContainer" containerID="8a01247d2390af21b812ee0740e4cf6fb80482d3a2ab5bcb4787f9bbd8d98910"
Sep 11 01:24:05.453276 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:05.453258    2574 scope.go:117] "RemoveContainer" containerID="7f22573b6d59bc474147fdb44e222b292e36c376c4efebc124da8eb9d83f7967"
Sep 11 01:24:05.460220 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:05.460199    2574 scope.go:117] "RemoveContainer" containerID="d5b5212ca7841bbe516845c0281cd1a2352e8322e592a8ff288f112b5930d1dd"
Sep 11 01:24:11.133947 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:11.133896    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:24:11.233663 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:11.233633    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:24:12.137166 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:12.137108    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:24:14.645121 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:14.645085    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:24:15.542679 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:15.542643    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:24:18.077304 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:18.077270    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:24:21.334439 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:21.334400    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:24:21.433618 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:21.433584    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:24:21.712199 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:21.712162    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:24:22.031554 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:22.031470    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:24:22.154118 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:22.154091    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:24:25.080616 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:25.080584    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:24:25.225353 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:25.225321    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:24:26.338443 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:26.338404    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:24:26.438742 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:26.438709    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:24:26.837081 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:26.837049    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:24:26.915672 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:26.915638    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:24:26.944260 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:26.944231    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f729f8-57b47d7b6-7brc8"]
Sep 11 01:24:26.944617 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:26.944605    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c67d64bb-40f3-4540-a3a0-8001a339a21c" containerName="payload-pre-processing"
Sep 11 01:24:26.944662 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:26.944620    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c67d64bb-40f3-4540-a3a0-8001a339a21c" containerName="payload-pre-processing"
Sep 11 01:24:26.944662 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:26.944628    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f75a0b21-b351-4447-8774-e7b34f3fd258" containerName="main"
Sep 11 01:24:26.944662 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:26.944635    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f75a0b21-b351-4447-8774-e7b34f3fd258" containerName="main"
Sep 11 01:24:26.944662 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:26.944641    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7159c6eb-0eac-492d-a5a0-720156e16245" containerName="main"
Sep 11 01:24:26.944662 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:26.944646    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7159c6eb-0eac-492d-a5a0-720156e16245" containerName="main"
Sep 11 01:24:26.944662 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:26.944658    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4337d7d-b9d3-43f4-9cb8-5ae74419c273" containerName="payload-processing"
Sep 11 01:24:26.944662 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:26.944664    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4337d7d-b9d3-43f4-9cb8-5ae74419c273" containerName="payload-processing"
Sep 11 01:24:26.944943 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:26.944672    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99d4765d-2ce1-4c6d-b70d-9fa16b0e59c2" containerName="revoke-keys"
Sep 11 01:24:26.944943 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:26.944679    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="99d4765d-2ce1-4c6d-b70d-9fa16b0e59c2" containerName="revoke-keys"
Sep 11 01:24:26.944943 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:26.944692    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e8b3e1d-7599-4c9f-bea1-e921bbb11d91" containerName="main"
Sep 11 01:24:26.944943 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:26.944697    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e8b3e1d-7599-4c9f-bea1-e921bbb11d91" containerName="main"
Sep 11 01:24:26.944943 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:26.944754    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f75a0b21-b351-4447-8774-e7b34f3fd258" containerName="main"
Sep 11 01:24:26.944943 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:26.944762    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e8b3e1d-7599-4c9f-bea1-e921bbb11d91" containerName="main"
Sep 11 01:24:26.944943 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:26.944768    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4337d7d-b9d3-43f4-9cb8-5ae74419c273" containerName="payload-processing"
Sep 11 01:24:26.944943 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:26.944774    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="99d4765d-2ce1-4c6d-b70d-9fa16b0e59c2" containerName="revoke-keys"
Sep 11 01:24:26.944943 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:26.944779    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7159c6eb-0eac-492d-a5a0-720156e16245" containerName="main"
Sep 11 01:24:26.944943 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:26.944784    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c67d64bb-40f3-4540-a3a0-8001a339a21c" containerName="payload-pre-processing"
Sep 11 01:24:26.949455 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:26.949437    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f729f8-57b47d7b6-7brc8"
Sep 11 01:24:26.951782 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:26.951763    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-f729f8\""
Sep 11 01:24:26.951963 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:26.951946    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-f729f8-dockercfg-2s5h4\""
Sep 11 01:24:26.956276 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:26.956122    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f729f8-fbb9d5d9f-dhb4r"]
Sep 11 01:24:26.959718 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:26.959697    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f729f8-57b47d7b6-7brc8"]
Sep 11 01:24:26.959801 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:26.959789    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-f729f8-fbb9d5d9f-dhb4r"
Sep 11 01:24:26.970064 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:26.970045    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f729f8-fbb9d5d9f-dhb4r"]
Sep 11 01:24:26.984291 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:26.984266    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p54z8\" (UniqueName: \"kubernetes.io/projected/f9450a23-e68a-417b-aebd-5ea1b3db935b-kube-api-access-p54z8\") pod \"payload-pre-processing-e2e-api-a-f729f8-57b47d7b6-7brc8\" (UID: \"f9450a23-e68a-417b-aebd-5ea1b3db935b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f729f8-57b47d7b6-7brc8"
Sep 11 01:24:26.984459 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:26.984306    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b-plugins-config-volume\") pod \"payload-processing-e2e-api-a-f729f8-fbb9d5d9f-dhb4r\" (UID: \"0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b\") " pod="openshift-ingress/payload-processing-e2e-api-a-f729f8-fbb9d5d9f-dhb4r"
Sep 11 01:24:26.984459 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:26.984335    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9450a23-e68a-417b-aebd-5ea1b3db935b-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-f729f8-57b47d7b6-7brc8\" (UID: \"f9450a23-e68a-417b-aebd-5ea1b3db935b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f729f8-57b47d7b6-7brc8"
Sep 11 01:24:26.984574 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:26.984460    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qv72j\" (UniqueName: \"kubernetes.io/projected/0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b-kube-api-access-qv72j\") pod \"payload-processing-e2e-api-a-f729f8-fbb9d5d9f-dhb4r\" (UID: \"0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b\") " pod="openshift-ingress/payload-processing-e2e-api-a-f729f8-fbb9d5d9f-dhb4r"
Sep 11 01:24:27.085338 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:27.085305    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv72j\" (UniqueName: \"kubernetes.io/projected/0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b-kube-api-access-qv72j\") pod \"payload-processing-e2e-api-a-f729f8-fbb9d5d9f-dhb4r\" (UID: \"0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b\") " pod="openshift-ingress/payload-processing-e2e-api-a-f729f8-fbb9d5d9f-dhb4r"
Sep 11 01:24:27.085498 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:27.085368    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p54z8\" (UniqueName: \"kubernetes.io/projected/f9450a23-e68a-417b-aebd-5ea1b3db935b-kube-api-access-p54z8\") pod \"payload-pre-processing-e2e-api-a-f729f8-57b47d7b6-7brc8\" (UID: \"f9450a23-e68a-417b-aebd-5ea1b3db935b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f729f8-57b47d7b6-7brc8"
Sep 11 01:24:27.085498 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:27.085389    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b-plugins-config-volume\") pod \"payload-processing-e2e-api-a-f729f8-fbb9d5d9f-dhb4r\" (UID: \"0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b\") " pod="openshift-ingress/payload-processing-e2e-api-a-f729f8-fbb9d5d9f-dhb4r"
Sep 11 01:24:27.085498 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:27.085417    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9450a23-e68a-417b-aebd-5ea1b3db935b-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-f729f8-57b47d7b6-7brc8\" (UID: \"f9450a23-e68a-417b-aebd-5ea1b3db935b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f729f8-57b47d7b6-7brc8"
Sep 11 01:24:27.086118 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:27.086097    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b-plugins-config-volume\") pod \"payload-processing-e2e-api-a-f729f8-fbb9d5d9f-dhb4r\" (UID: \"0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b\") " pod="openshift-ingress/payload-processing-e2e-api-a-f729f8-fbb9d5d9f-dhb4r"
Sep 11 01:24:27.086199 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:27.086136    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9450a23-e68a-417b-aebd-5ea1b3db935b-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-f729f8-57b47d7b6-7brc8\" (UID: \"f9450a23-e68a-417b-aebd-5ea1b3db935b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f729f8-57b47d7b6-7brc8"
Sep 11 01:24:27.094131 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:27.094065    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qv72j\" (UniqueName: \"kubernetes.io/projected/0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b-kube-api-access-qv72j\") pod \"payload-processing-e2e-api-a-f729f8-fbb9d5d9f-dhb4r\" (UID: \"0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b\") " pod="openshift-ingress/payload-processing-e2e-api-a-f729f8-fbb9d5d9f-dhb4r"
Sep 11 01:24:27.094241 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:27.094123    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p54z8\" (UniqueName: \"kubernetes.io/projected/f9450a23-e68a-417b-aebd-5ea1b3db935b-kube-api-access-p54z8\") pod \"payload-pre-processing-e2e-api-a-f729f8-57b47d7b6-7brc8\" (UID: \"f9450a23-e68a-417b-aebd-5ea1b3db935b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f729f8-57b47d7b6-7brc8"
Sep 11 01:24:27.261077 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:27.261048    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f729f8-57b47d7b6-7brc8"
Sep 11 01:24:27.271103 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:27.271077    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-f729f8-fbb9d5d9f-dhb4r"
Sep 11 01:24:27.538294 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:27.538258    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f729f8-57b47d7b6-7brc8"]
Sep 11 01:24:27.540040 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:24:27.540019    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9450a23_e68a_417b_aebd_5ea1b3db935b.slice/crio-62db86360c48874059b9b9469e55acd9d6585d86ed01ca3c4d6e55f186fd5a6b WatchSource:0}: Error finding container 62db86360c48874059b9b9469e55acd9d6585d86ed01ca3c4d6e55f186fd5a6b: Status 404 returned error can't find the container with id 62db86360c48874059b9b9469e55acd9d6585d86ed01ca3c4d6e55f186fd5a6b
Sep 11 01:24:27.550392 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:27.550365    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f729f8-fbb9d5d9f-dhb4r"]
Sep 11 01:24:27.553403 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:24:27.553385    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a0c3c1c_494d_447b_b5a8_4928ba9f2a0b.slice/crio-a730353ecf3bf84181c898a16cb8302540ff66fb6a3ec709dda1e80bfad4955e WatchSource:0}: Error finding container a730353ecf3bf84181c898a16cb8302540ff66fb6a3ec709dda1e80bfad4955e: Status 404 returned error can't find the container with id a730353ecf3bf84181c898a16cb8302540ff66fb6a3ec709dda1e80bfad4955e
Sep 11 01:24:27.893985 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:27.893869    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-f729f8-fbb9d5d9f-dhb4r" event={"ID":"0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b","Type":"ContainerStarted","Data":"73b008ba7de879313df468b7fa13a2272519722e9bce6a724d85c5de8af7db29"}
Sep 11 01:24:27.893985 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:27.893948    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-f729f8-fbb9d5d9f-dhb4r" event={"ID":"0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b","Type":"ContainerStarted","Data":"a730353ecf3bf84181c898a16cb8302540ff66fb6a3ec709dda1e80bfad4955e"}
Sep 11 01:24:27.894219 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:27.894050    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-f729f8-fbb9d5d9f-dhb4r"
Sep 11 01:24:27.895779 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:27.895747    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f729f8-57b47d7b6-7brc8" event={"ID":"f9450a23-e68a-417b-aebd-5ea1b3db935b","Type":"ContainerStarted","Data":"f6b0a015c398777ddac723e374f36bc125d9f4c0fc726b03e3c398f87ee4c59c"}
Sep 11 01:24:27.895899 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:27.895797    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f729f8-57b47d7b6-7brc8" event={"ID":"f9450a23-e68a-417b-aebd-5ea1b3db935b","Type":"ContainerStarted","Data":"62db86360c48874059b9b9469e55acd9d6585d86ed01ca3c4d6e55f186fd5a6b"}
Sep 11 01:24:27.895899 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:27.895826    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f729f8-57b47d7b6-7brc8"
Sep 11 01:24:27.916680 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:27.916640    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-f729f8-fbb9d5d9f-dhb4r" podStartSLOduration=1.9166264050000001 podStartE2EDuration="1.916626405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:24:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:24:27.915462665 +0000 UTC m=+1583.312654842" watchObservedRunningTime="2026-09-11 01:24:27.916626405 +0000 UTC m=+1583.313818578"
Sep 11 01:24:27.941290 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:27.941247    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f729f8-57b47d7b6-7brc8" podStartSLOduration=1.9412367160000001 podStartE2EDuration="1.941236716s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:24:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:24:27.939674154 +0000 UTC m=+1583.336866338" watchObservedRunningTime="2026-09-11 01:24:27.941236716 +0000 UTC m=+1583.338428889"
Sep 11 01:24:38.902664 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:38.902637    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f729f8-57b47d7b6-7brc8"
Sep 11 01:24:38.903099 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:38.902802    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-f729f8-fbb9d5d9f-dhb4r"
Sep 11 01:24:50.061635 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:50.061599    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:24:50.532667 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:50.532634    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:24:50.842616 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:50.842536    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:24:51.064980 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:51.064943    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:24:54.258113 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:54.258084    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:24:54.358005 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:54.357967    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:24:55.426088 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:55.426035    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:24:55.536258 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:55.536223    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:24:55.847716 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:55.847632    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:24:55.949090 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:55.949054    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:24:56.020456 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:56.020426    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:24:56.025435 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:56.025411    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-be588a-6cdffc5cf8-xnhx6"]
Sep 11 01:24:56.028901 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:56.028884    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-be588a-6cdffc5cf8-xnhx6"
Sep 11 01:24:56.033824 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:56.033806    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-be588a\""
Sep 11 01:24:56.033920 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:56.033810    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-be588a-dockercfg-4qsjq\""
Sep 11 01:24:56.053068 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:56.053042    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-be588a-6cdffc5cf8-xnhx6"]
Sep 11 01:24:56.069836 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:56.069806    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-be588a-65d9794cf-9b2d7"]
Sep 11 01:24:56.073156 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:56.073138    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-be588a-65d9794cf-9b2d7"
Sep 11 01:24:56.090506 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:56.085758    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-be588a-65d9794cf-9b2d7"]
Sep 11 01:24:56.136667 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:56.136584    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-be588a-6cdffc5cf8-xnhx6\" (UID: \"b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-be588a-6cdffc5cf8-xnhx6"
Sep 11 01:24:56.136667 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:56.136624    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9slh\" (UniqueName: \"kubernetes.io/projected/b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e-kube-api-access-m9slh\") pod \"payload-pre-processing-e2e-api-b-be588a-6cdffc5cf8-xnhx6\" (UID: \"b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-be588a-6cdffc5cf8-xnhx6"
Sep 11 01:24:56.237698 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:56.237655    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-be588a-6cdffc5cf8-xnhx6\" (UID: \"b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-be588a-6cdffc5cf8-xnhx6"
Sep 11 01:24:56.237850 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:56.237710    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9slh\" (UniqueName: \"kubernetes.io/projected/b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e-kube-api-access-m9slh\") pod \"payload-pre-processing-e2e-api-b-be588a-6cdffc5cf8-xnhx6\" (UID: \"b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-be588a-6cdffc5cf8-xnhx6"
Sep 11 01:24:56.237850 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:56.237762    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhpv6\" (UniqueName: \"kubernetes.io/projected/158ac01d-720e-4d0e-9d99-26fa0cf9f086-kube-api-access-vhpv6\") pod \"payload-processing-e2e-api-b-be588a-65d9794cf-9b2d7\" (UID: \"158ac01d-720e-4d0e-9d99-26fa0cf9f086\") " pod="openshift-ingress/payload-processing-e2e-api-b-be588a-65d9794cf-9b2d7"
Sep 11 01:24:56.237850 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:56.237784    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/158ac01d-720e-4d0e-9d99-26fa0cf9f086-plugins-config-volume\") pod \"payload-processing-e2e-api-b-be588a-65d9794cf-9b2d7\" (UID: \"158ac01d-720e-4d0e-9d99-26fa0cf9f086\") " pod="openshift-ingress/payload-processing-e2e-api-b-be588a-65d9794cf-9b2d7"
Sep 11 01:24:56.238471 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:56.238451    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-be588a-6cdffc5cf8-xnhx6\" (UID: \"b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-be588a-6cdffc5cf8-xnhx6"
Sep 11 01:24:56.246798 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:56.246776    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9slh\" (UniqueName: \"kubernetes.io/projected/b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e-kube-api-access-m9slh\") pod \"payload-pre-processing-e2e-api-b-be588a-6cdffc5cf8-xnhx6\" (UID: \"b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-be588a-6cdffc5cf8-xnhx6"
Sep 11 01:24:56.339228 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:56.339162    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhpv6\" (UniqueName: \"kubernetes.io/projected/158ac01d-720e-4d0e-9d99-26fa0cf9f086-kube-api-access-vhpv6\") pod \"payload-processing-e2e-api-b-be588a-65d9794cf-9b2d7\" (UID: \"158ac01d-720e-4d0e-9d99-26fa0cf9f086\") " pod="openshift-ingress/payload-processing-e2e-api-b-be588a-65d9794cf-9b2d7"
Sep 11 01:24:56.339228 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:56.339199    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/158ac01d-720e-4d0e-9d99-26fa0cf9f086-plugins-config-volume\") pod \"payload-processing-e2e-api-b-be588a-65d9794cf-9b2d7\" (UID: \"158ac01d-720e-4d0e-9d99-26fa0cf9f086\") " pod="openshift-ingress/payload-processing-e2e-api-b-be588a-65d9794cf-9b2d7"
Sep 11 01:24:56.339419 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:56.339225    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-be588a-6cdffc5cf8-xnhx6"
Sep 11 01:24:56.339821 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:56.339803    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/158ac01d-720e-4d0e-9d99-26fa0cf9f086-plugins-config-volume\") pod \"payload-processing-e2e-api-b-be588a-65d9794cf-9b2d7\" (UID: \"158ac01d-720e-4d0e-9d99-26fa0cf9f086\") " pod="openshift-ingress/payload-processing-e2e-api-b-be588a-65d9794cf-9b2d7"
Sep 11 01:24:56.347422 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:56.347392    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhpv6\" (UniqueName: \"kubernetes.io/projected/158ac01d-720e-4d0e-9d99-26fa0cf9f086-kube-api-access-vhpv6\") pod \"payload-processing-e2e-api-b-be588a-65d9794cf-9b2d7\" (UID: \"158ac01d-720e-4d0e-9d99-26fa0cf9f086\") " pod="openshift-ingress/payload-processing-e2e-api-b-be588a-65d9794cf-9b2d7"
Sep 11 01:24:56.383420 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:56.383389    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-be588a-65d9794cf-9b2d7"
Sep 11 01:24:56.611001 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:56.610976    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-be588a-6cdffc5cf8-xnhx6"]
Sep 11 01:24:56.612693 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:24:56.612671    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb55bf1ad_c538_4be5_87c4_bd32ed6d9c8e.slice/crio-30b674b323d2ca3e7e191e3eae37cb2d485fe70234523de0acf163f9a3057885 WatchSource:0}: Error finding container 30b674b323d2ca3e7e191e3eae37cb2d485fe70234523de0acf163f9a3057885: Status 404 returned error can't find the container with id 30b674b323d2ca3e7e191e3eae37cb2d485fe70234523de0acf163f9a3057885
Sep 11 01:24:56.663160 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:56.663138    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-be588a-65d9794cf-9b2d7"]
Sep 11 01:24:56.678893 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:24:56.678872    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod158ac01d_720e_4d0e_9d99_26fa0cf9f086.slice/crio-c885741cb7617bdf830501f2269b4eb8f72e674ccae886a92029c5f8b8f7ce2c WatchSource:0}: Error finding container c885741cb7617bdf830501f2269b4eb8f72e674ccae886a92029c5f8b8f7ce2c: Status 404 returned error can't find the container with id c885741cb7617bdf830501f2269b4eb8f72e674ccae886a92029c5f8b8f7ce2c
Sep 11 01:24:57.038860 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:57.038817    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-be588a-6cdffc5cf8-xnhx6" event={"ID":"b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e","Type":"ContainerStarted","Data":"c9ea480ee2db714dc5176306772ef9a9ce05c69d7bce9033fca1eadab086cefc"}
Sep 11 01:24:57.039100 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:57.038899    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-be588a-6cdffc5cf8-xnhx6"
Sep 11 01:24:57.039100 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:57.038942    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-be588a-6cdffc5cf8-xnhx6" event={"ID":"b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e","Type":"ContainerStarted","Data":"30b674b323d2ca3e7e191e3eae37cb2d485fe70234523de0acf163f9a3057885"}
Sep 11 01:24:57.040362 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:57.040339    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-be588a-65d9794cf-9b2d7" event={"ID":"158ac01d-720e-4d0e-9d99-26fa0cf9f086","Type":"ContainerStarted","Data":"5c129acc1571be4272d70ea656545f181054f6ac555483219452f0f2b56abd46"}
Sep 11 01:24:57.040479 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:57.040385    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-be588a-65d9794cf-9b2d7" event={"ID":"158ac01d-720e-4d0e-9d99-26fa0cf9f086","Type":"ContainerStarted","Data":"c885741cb7617bdf830501f2269b4eb8f72e674ccae886a92029c5f8b8f7ce2c"}
Sep 11 01:24:57.040479 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:57.040415    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-be588a-65d9794cf-9b2d7"
Sep 11 01:24:57.054623 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:57.054555    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-be588a-6cdffc5cf8-xnhx6" podStartSLOduration=1.054536841 podStartE2EDuration="1.054536841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:24:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:24:57.05300583 +0000 UTC m=+1612.450198014" watchObservedRunningTime="2026-09-11 01:24:57.054536841 +0000 UTC m=+1612.451729022"
Sep 11 01:24:57.068169 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:24:57.068127    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-be588a-65d9794cf-9b2d7" podStartSLOduration=1.0681124419999999 podStartE2EDuration="1.068112442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:24:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:24:57.066590594 +0000 UTC m=+1612.463782766" watchObservedRunningTime="2026-09-11 01:24:57.068112442 +0000 UTC m=+1612.465304614"
Sep 11 01:25:08.049546 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:08.049510    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-be588a-65d9794cf-9b2d7"
Sep 11 01:25:08.049987 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:08.049692    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-be588a-6cdffc5cf8-xnhx6"
Sep 11 01:25:19.337196 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:19.337161    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:25:22.845227 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:22.845191    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f729f8-rnltn"]
Sep 11 01:25:22.848855 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:22.848836    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f729f8-rnltn"
Sep 11 01:25:22.851001 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:22.850973    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 01:25:22.851118 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:22.851000    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q6gbn\""
Sep 11 01:25:22.855461 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:22.855439    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f729f8-rnltn"]
Sep 11 01:25:22.938876 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:22.938812    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:25:22.968779 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:22.968751    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdf6d4b5-dddb-46e3-9c3e-530e01630ca7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-f729f8-rnltn\" (UID: \"fdf6d4b5-dddb-46e3-9c3e-530e01630ca7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f729f8-rnltn"
Sep 11 01:25:23.069239 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:23.069199    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdf6d4b5-dddb-46e3-9c3e-530e01630ca7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-f729f8-rnltn\" (UID: \"fdf6d4b5-dddb-46e3-9c3e-530e01630ca7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f729f8-rnltn"
Sep 11 01:25:23.069805 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:23.069785    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdf6d4b5-dddb-46e3-9c3e-530e01630ca7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-f729f8-rnltn\" (UID: \"fdf6d4b5-dddb-46e3-9c3e-530e01630ca7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f729f8-rnltn"
Sep 11 01:25:23.161002 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:23.160984    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f729f8-rnltn"
Sep 11 01:25:23.375622 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:23.375585    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:25:23.433389 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:23.433367    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f729f8-rnltn"]
Sep 11 01:25:23.435492 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:25:23.435462    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdf6d4b5_dddb_46e3_9c3e_530e01630ca7.slice/crio-e46c9678f82d7c4fffb38db31811cb8a39640df78778e6cd39694108396a36e4 WatchSource:0}: Error finding container e46c9678f82d7c4fffb38db31811cb8a39640df78778e6cd39694108396a36e4: Status 404 returned error can't find the container with id e46c9678f82d7c4fffb38db31811cb8a39640df78778e6cd39694108396a36e4
Sep 11 01:25:23.435948 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:23.435899    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:25:24.185947 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:24.185900    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f729f8-rnltn" event={"ID":"fdf6d4b5-dddb-46e3-9c3e-530e01630ca7","Type":"ContainerStarted","Data":"6053c1ab13feeb80d53f7e094a57497b8aa5bce45a881da67a6e5685989e0e99"}
Sep 11 01:25:24.186295 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:24.185982    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f729f8-rnltn" event={"ID":"fdf6d4b5-dddb-46e3-9c3e-530e01630ca7","Type":"ContainerStarted","Data":"e46c9678f82d7c4fffb38db31811cb8a39640df78778e6cd39694108396a36e4"}
Sep 11 01:25:25.192039 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:25.192006    2574 generic.go:356] "Generic (PLEG): container finished" podID="fdf6d4b5-dddb-46e3-9c3e-530e01630ca7" containerID="6053c1ab13feeb80d53f7e094a57497b8aa5bce45a881da67a6e5685989e0e99" exitCode=0
Sep 11 01:25:25.192499 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:25.192051    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f729f8-rnltn" event={"ID":"fdf6d4b5-dddb-46e3-9c3e-530e01630ca7","Type":"ContainerDied","Data":"6053c1ab13feeb80d53f7e094a57497b8aa5bce45a881da67a6e5685989e0e99"}
Sep 11 01:25:25.322638 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:25.322616    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f729f8-rnltn"
Sep 11 01:25:25.390548 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:25.390519    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdf6d4b5-dddb-46e3-9c3e-530e01630ca7-maas-api-service-ca\") pod \"fdf6d4b5-dddb-46e3-9c3e-530e01630ca7\" (UID: \"fdf6d4b5-dddb-46e3-9c3e-530e01630ca7\") "
Sep 11 01:25:25.390892 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:25.390871    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdf6d4b5-dddb-46e3-9c3e-530e01630ca7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fdf6d4b5-dddb-46e3-9c3e-530e01630ca7" (UID: "fdf6d4b5-dddb-46e3-9c3e-530e01630ca7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:25:25.491418 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:25.491360    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdf6d4b5-dddb-46e3-9c3e-530e01630ca7-maas-api-service-ca\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:25:26.198718 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:26.198685    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f729f8-rnltn" event={"ID":"fdf6d4b5-dddb-46e3-9c3e-530e01630ca7","Type":"ContainerDied","Data":"e46c9678f82d7c4fffb38db31811cb8a39640df78778e6cd39694108396a36e4"}
Sep 11 01:25:26.198718 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:26.198708    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f729f8-rnltn"
Sep 11 01:25:26.199160 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:26.198731    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e46c9678f82d7c4fffb38db31811cb8a39640df78778e6cd39694108396a36e4"
Sep 11 01:25:38.475506 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:38.475473    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:25:38.500864 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:38.500828    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f729f8-fbb9d5d9f-dhb4r"]
Sep 11 01:25:38.501129 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:38.501100    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-f729f8-fbb9d5d9f-dhb4r" podUID="0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b" containerName="payload-processing" containerID="cri-o://73b008ba7de879313df468b7fa13a2272519722e9bce6a724d85c5de8af7db29" gracePeriod=30
Sep 11 01:25:38.511109 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:38.511081    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f729f8-57b47d7b6-7brc8"]
Sep 11 01:25:38.511498 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:38.511452    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f729f8-57b47d7b6-7brc8" podUID="f9450a23-e68a-417b-aebd-5ea1b3db935b" containerName="payload-pre-processing" containerID="cri-o://f6b0a015c398777ddac723e374f36bc125d9f4c0fc726b03e3c398f87ee4c59c" gracePeriod=30
Sep 11 01:25:38.572720 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:38.572688    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:25:38.677170 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:38.677127    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:25:38.683797 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:38.683771    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f729f8-rnltn"]
Sep 11 01:25:38.687753 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:38.687729    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f729f8-rnltn"]
Sep 11 01:25:38.739322 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:38.739255    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:25:38.788272 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:38.788254    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-f729f8-fbb9d5d9f-dhb4r"
Sep 11 01:25:38.793046 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:38.793027    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f729f8-57b47d7b6-7brc8"
Sep 11 01:25:38.924016 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:38.923984    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qv72j\" (UniqueName: \"kubernetes.io/projected/0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b-kube-api-access-qv72j\") pod \"0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b\" (UID: \"0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b\") "
Sep 11 01:25:38.924169 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:38.924030    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p54z8\" (UniqueName: \"kubernetes.io/projected/f9450a23-e68a-417b-aebd-5ea1b3db935b-kube-api-access-p54z8\") pod \"f9450a23-e68a-417b-aebd-5ea1b3db935b\" (UID: \"f9450a23-e68a-417b-aebd-5ea1b3db935b\") "
Sep 11 01:25:38.924169 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:38.924066    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b-plugins-config-volume\") pod \"0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b\" (UID: \"0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b\") "
Sep 11 01:25:38.924169 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:38.924125    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9450a23-e68a-417b-aebd-5ea1b3db935b-plugins-config-volume\") pod \"f9450a23-e68a-417b-aebd-5ea1b3db935b\" (UID: \"f9450a23-e68a-417b-aebd-5ea1b3db935b\") "
Sep 11 01:25:38.924497 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:38.924467    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b" (UID: "0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:25:38.924655 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:38.924506    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9450a23-e68a-417b-aebd-5ea1b3db935b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f9450a23-e68a-417b-aebd-5ea1b3db935b" (UID: "f9450a23-e68a-417b-aebd-5ea1b3db935b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:25:38.926273 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:38.926252    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b-kube-api-access-qv72j" (OuterVolumeSpecName: "kube-api-access-qv72j") pod "0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b" (UID: "0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b"). InnerVolumeSpecName "kube-api-access-qv72j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:25:38.926376 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:38.926286    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9450a23-e68a-417b-aebd-5ea1b3db935b-kube-api-access-p54z8" (OuterVolumeSpecName: "kube-api-access-p54z8") pod "f9450a23-e68a-417b-aebd-5ea1b3db935b" (UID: "f9450a23-e68a-417b-aebd-5ea1b3db935b"). InnerVolumeSpecName "kube-api-access-p54z8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:25:39.025245 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:39.025187    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9450a23-e68a-417b-aebd-5ea1b3db935b-plugins-config-volume\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:25:39.025245 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:39.025208    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qv72j\" (UniqueName: \"kubernetes.io/projected/0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b-kube-api-access-qv72j\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:25:39.025245 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:39.025218    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p54z8\" (UniqueName: \"kubernetes.io/projected/f9450a23-e68a-417b-aebd-5ea1b3db935b-kube-api-access-p54z8\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:25:39.025245 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:39.025226    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b-plugins-config-volume\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:25:39.091270 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:39.091248    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:25:39.100266 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:39.100239    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdf6d4b5-dddb-46e3-9c3e-530e01630ca7" path="/var/lib/kubelet/pods/fdf6d4b5-dddb-46e3-9c3e-530e01630ca7/volumes"
Sep 11 01:25:39.131584 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:25:39.131554    2574 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a0c3c1c_494d_447b_b5a8_4928ba9f2a0b.slice\": RecentStats: unable to find data in memory cache]"
Sep 11 01:25:39.268600 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:39.268567    2574 generic.go:356] "Generic (PLEG): container finished" podID="0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b" containerID="73b008ba7de879313df468b7fa13a2272519722e9bce6a724d85c5de8af7db29" exitCode=0
Sep 11 01:25:39.268806 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:39.268618    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-f729f8-fbb9d5d9f-dhb4r"
Sep 11 01:25:39.268806 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:39.268654    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-f729f8-fbb9d5d9f-dhb4r" event={"ID":"0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b","Type":"ContainerDied","Data":"73b008ba7de879313df468b7fa13a2272519722e9bce6a724d85c5de8af7db29"}
Sep 11 01:25:39.268806 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:39.268705    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-f729f8-fbb9d5d9f-dhb4r" event={"ID":"0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b","Type":"ContainerDied","Data":"a730353ecf3bf84181c898a16cb8302540ff66fb6a3ec709dda1e80bfad4955e"}
Sep 11 01:25:39.268806 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:39.268727    2574 scope.go:117] "RemoveContainer" containerID="73b008ba7de879313df468b7fa13a2272519722e9bce6a724d85c5de8af7db29"
Sep 11 01:25:39.270017 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:39.269994    2574 generic.go:356] "Generic (PLEG): container finished" podID="f9450a23-e68a-417b-aebd-5ea1b3db935b" containerID="f6b0a015c398777ddac723e374f36bc125d9f4c0fc726b03e3c398f87ee4c59c" exitCode=0
Sep 11 01:25:39.270129 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:39.270034    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f729f8-57b47d7b6-7brc8"
Sep 11 01:25:39.270129 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:39.270057    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f729f8-57b47d7b6-7brc8" event={"ID":"f9450a23-e68a-417b-aebd-5ea1b3db935b","Type":"ContainerDied","Data":"f6b0a015c398777ddac723e374f36bc125d9f4c0fc726b03e3c398f87ee4c59c"}
Sep 11 01:25:39.270129 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:39.270090    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f729f8-57b47d7b6-7brc8" event={"ID":"f9450a23-e68a-417b-aebd-5ea1b3db935b","Type":"ContainerDied","Data":"62db86360c48874059b9b9469e55acd9d6585d86ed01ca3c4d6e55f186fd5a6b"}
Sep 11 01:25:39.278152 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:39.278092    2574 scope.go:117] "RemoveContainer" containerID="73b008ba7de879313df468b7fa13a2272519722e9bce6a724d85c5de8af7db29"
Sep 11 01:25:39.278372 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:25:39.278355    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73b008ba7de879313df468b7fa13a2272519722e9bce6a724d85c5de8af7db29\": container with ID starting with 73b008ba7de879313df468b7fa13a2272519722e9bce6a724d85c5de8af7db29 not found: ID does not exist" containerID="73b008ba7de879313df468b7fa13a2272519722e9bce6a724d85c5de8af7db29"
Sep 11 01:25:39.278419 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:39.278379    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73b008ba7de879313df468b7fa13a2272519722e9bce6a724d85c5de8af7db29"} err="failed to get container status \"73b008ba7de879313df468b7fa13a2272519722e9bce6a724d85c5de8af7db29\": rpc error: code = NotFound desc = could not find container \"73b008ba7de879313df468b7fa13a2272519722e9bce6a724d85c5de8af7db29\": container with ID starting with 73b008ba7de879313df468b7fa13a2272519722e9bce6a724d85c5de8af7db29 not found: ID does not exist"
Sep 11 01:25:39.278419 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:39.278394    2574 scope.go:117] "RemoveContainer" containerID="f6b0a015c398777ddac723e374f36bc125d9f4c0fc726b03e3c398f87ee4c59c"
Sep 11 01:25:39.287763 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:39.287383    2574 scope.go:117] "RemoveContainer" containerID="f6b0a015c398777ddac723e374f36bc125d9f4c0fc726b03e3c398f87ee4c59c"
Sep 11 01:25:39.287867 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:25:39.287809    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6b0a015c398777ddac723e374f36bc125d9f4c0fc726b03e3c398f87ee4c59c\": container with ID starting with f6b0a015c398777ddac723e374f36bc125d9f4c0fc726b03e3c398f87ee4c59c not found: ID does not exist" containerID="f6b0a015c398777ddac723e374f36bc125d9f4c0fc726b03e3c398f87ee4c59c"
Sep 11 01:25:39.287867 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:39.287847    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6b0a015c398777ddac723e374f36bc125d9f4c0fc726b03e3c398f87ee4c59c"} err="failed to get container status \"f6b0a015c398777ddac723e374f36bc125d9f4c0fc726b03e3c398f87ee4c59c\": rpc error: code = NotFound desc = could not find container \"f6b0a015c398777ddac723e374f36bc125d9f4c0fc726b03e3c398f87ee4c59c\": container with ID starting with f6b0a015c398777ddac723e374f36bc125d9f4c0fc726b03e3c398f87ee4c59c not found: ID does not exist"
Sep 11 01:25:39.288444 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:39.288420    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f729f8-fbb9d5d9f-dhb4r"]
Sep 11 01:25:39.294080 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:39.294056    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f729f8-fbb9d5d9f-dhb4r"]
Sep 11 01:25:39.303775 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:39.303751    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f729f8-57b47d7b6-7brc8"]
Sep 11 01:25:39.310381 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:39.310357    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f729f8-57b47d7b6-7brc8"]
Sep 11 01:25:41.100023 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:41.099987    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b" path="/var/lib/kubelet/pods/0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b/volumes"
Sep 11 01:25:41.100426 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:41.100338    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9450a23-e68a-417b-aebd-5ea1b3db935b" path="/var/lib/kubelet/pods/f9450a23-e68a-417b-aebd-5ea1b3db935b/volumes"
Sep 11 01:25:44.726198 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:44.726160    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:25:44.787096 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:44.787060    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:25:45.041786 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:45.041678    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:25:45.457956 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:45.457921    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-be588a-fvq92"]
Sep 11 01:25:45.458421 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:45.458402    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9450a23-e68a-417b-aebd-5ea1b3db935b" containerName="payload-pre-processing"
Sep 11 01:25:45.458503 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:45.458426    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9450a23-e68a-417b-aebd-5ea1b3db935b" containerName="payload-pre-processing"
Sep 11 01:25:45.458503 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:45.458456    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdf6d4b5-dddb-46e3-9c3e-530e01630ca7" containerName="revoke-keys"
Sep 11 01:25:45.458503 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:45.458467    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdf6d4b5-dddb-46e3-9c3e-530e01630ca7" containerName="revoke-keys"
Sep 11 01:25:45.458503 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:45.458485    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b" containerName="payload-processing"
Sep 11 01:25:45.458503 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:45.458494    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b" containerName="payload-processing"
Sep 11 01:25:45.458679 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:45.458582    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a0c3c1c-494d-447b-b5a8-4928ba9f2a0b" containerName="payload-processing"
Sep 11 01:25:45.458679 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:45.458594    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdf6d4b5-dddb-46e3-9c3e-530e01630ca7" containerName="revoke-keys"
Sep 11 01:25:45.458679 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:45.458604    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9450a23-e68a-417b-aebd-5ea1b3db935b" containerName="payload-pre-processing"
Sep 11 01:25:45.467087 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:45.467060    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-be588a-fvq92"
Sep 11 01:25:45.469349 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:45.469300    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 01:25:45.469494 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:45.469365    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q6gbn\""
Sep 11 01:25:45.469687 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:45.469643    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-be588a-fvq92"]
Sep 11 01:25:45.564289 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:45.564256    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:25:45.583471 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:45.583438    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fae00d0e-b85d-4666-b540-e823be346d07-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-be588a-fvq92\" (UID: \"fae00d0e-b85d-4666-b540-e823be346d07\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-be588a-fvq92"
Sep 11 01:25:45.684321 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:45.684289    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fae00d0e-b85d-4666-b540-e823be346d07-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-be588a-fvq92\" (UID: \"fae00d0e-b85d-4666-b540-e823be346d07\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-be588a-fvq92"
Sep 11 01:25:45.684850 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:45.684832    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fae00d0e-b85d-4666-b540-e823be346d07-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-be588a-fvq92\" (UID: \"fae00d0e-b85d-4666-b540-e823be346d07\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-be588a-fvq92"
Sep 11 01:25:45.781041 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:45.780960    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-be588a-fvq92"
Sep 11 01:25:46.053221 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:46.053177    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-be588a-fvq92"]
Sep 11 01:25:46.055782 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:25:46.055753    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfae00d0e_b85d_4666_b540_e823be346d07.slice/crio-b3dab96410b3751dde2de98d57648b2a987e2f466fd76829a5daad2fcb69cc7a WatchSource:0}: Error finding container b3dab96410b3751dde2de98d57648b2a987e2f466fd76829a5daad2fcb69cc7a: Status 404 returned error can't find the container with id b3dab96410b3751dde2de98d57648b2a987e2f466fd76829a5daad2fcb69cc7a
Sep 11 01:25:46.307640 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:46.307583    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-be588a-fvq92" event={"ID":"fae00d0e-b85d-4666-b540-e823be346d07","Type":"ContainerStarted","Data":"b3dab96410b3751dde2de98d57648b2a987e2f466fd76829a5daad2fcb69cc7a"}
Sep 11 01:25:47.314072 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:47.314037    2574 generic.go:356] "Generic (PLEG): container finished" podID="fae00d0e-b85d-4666-b540-e823be346d07" containerID="4f7ad6ddf73195d7fdcb12be117dedea4c59d76584a5cbc05ffef1ec0ac3da57" exitCode=0
Sep 11 01:25:47.314456 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:47.314115    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-be588a-fvq92" event={"ID":"fae00d0e-b85d-4666-b540-e823be346d07","Type":"ContainerDied","Data":"4f7ad6ddf73195d7fdcb12be117dedea4c59d76584a5cbc05ffef1ec0ac3da57"}
Sep 11 01:25:48.450829 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:48.450807    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-be588a-fvq92"
Sep 11 01:25:48.508487 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:48.508451    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fae00d0e-b85d-4666-b540-e823be346d07-maas-api-service-ca\") pod \"fae00d0e-b85d-4666-b540-e823be346d07\" (UID: \"fae00d0e-b85d-4666-b540-e823be346d07\") "
Sep 11 01:25:48.508799 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:48.508777    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fae00d0e-b85d-4666-b540-e823be346d07-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fae00d0e-b85d-4666-b540-e823be346d07" (UID: "fae00d0e-b85d-4666-b540-e823be346d07"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:25:48.609615 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:48.609547    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fae00d0e-b85d-4666-b540-e823be346d07-maas-api-service-ca\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:25:49.325219 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:49.325189    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-be588a-fvq92" event={"ID":"fae00d0e-b85d-4666-b540-e823be346d07","Type":"ContainerDied","Data":"b3dab96410b3751dde2de98d57648b2a987e2f466fd76829a5daad2fcb69cc7a"}
Sep 11 01:25:49.325219 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:49.325223    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-be588a-fvq92"
Sep 11 01:25:49.325394 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:49.325242    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b3dab96410b3751dde2de98d57648b2a987e2f466fd76829a5daad2fcb69cc7a"
Sep 11 01:25:54.316536 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:54.316506    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7965-762bb7"]
Sep 11 01:25:54.316931 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:54.316900    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fae00d0e-b85d-4666-b540-e823be346d07" containerName="revoke-keys"
Sep 11 01:25:54.316974 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:54.316932    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="fae00d0e-b85d-4666-b540-e823be346d07" containerName="revoke-keys"
Sep 11 01:25:54.317015 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:54.317001    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="fae00d0e-b85d-4666-b540-e823be346d07" containerName="revoke-keys"
Sep 11 01:25:54.320049 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:54.320032    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7965-762bb7"
Sep 11 01:25:54.322776 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:54.322752    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-7gdtv\""
Sep 11 01:25:54.324941 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:54.324899    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7965-762bb7"]
Sep 11 01:25:54.352596 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:54.352568    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpqfq\" (UniqueName: \"kubernetes.io/projected/0fdb04ad-e08a-4b27-aa43-5f5ec0e7c92c-kube-api-access-wpqfq\") pod \"test-curl-7965-762bb7\" (UID: \"0fdb04ad-e08a-4b27-aa43-5f5ec0e7c92c\") " pod="opendatahub/test-curl-7965-762bb7"
Sep 11 01:25:54.453217 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:54.453180    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpqfq\" (UniqueName: \"kubernetes.io/projected/0fdb04ad-e08a-4b27-aa43-5f5ec0e7c92c-kube-api-access-wpqfq\") pod \"test-curl-7965-762bb7\" (UID: \"0fdb04ad-e08a-4b27-aa43-5f5ec0e7c92c\") " pod="opendatahub/test-curl-7965-762bb7"
Sep 11 01:25:54.460667 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:54.460643    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpqfq\" (UniqueName: \"kubernetes.io/projected/0fdb04ad-e08a-4b27-aa43-5f5ec0e7c92c-kube-api-access-wpqfq\") pod \"test-curl-7965-762bb7\" (UID: \"0fdb04ad-e08a-4b27-aa43-5f5ec0e7c92c\") " pod="opendatahub/test-curl-7965-762bb7"
Sep 11 01:25:54.630386 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:54.630302    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7965-762bb7"
Sep 11 01:25:54.897398 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:54.897372    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7965-762bb7"]
Sep 11 01:25:54.899869 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:25:54.899845    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0fdb04ad_e08a_4b27_aa43_5f5ec0e7c92c.slice/crio-49b3ea256aa57501548d9413ee30b83976d1527daafc0f44ecba0aad3e2cec20 WatchSource:0}: Error finding container 49b3ea256aa57501548d9413ee30b83976d1527daafc0f44ecba0aad3e2cec20: Status 404 returned error can't find the container with id 49b3ea256aa57501548d9413ee30b83976d1527daafc0f44ecba0aad3e2cec20
Sep 11 01:25:55.359221 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:55.359184    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7965-762bb7" event={"ID":"0fdb04ad-e08a-4b27-aa43-5f5ec0e7c92c","Type":"ContainerStarted","Data":"13a2e531332576a9e0ce10a44e7413c289249def53fa5e76740d2ac9c2684ea1"}
Sep 11 01:25:55.359221 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:55.359222    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7965-762bb7" event={"ID":"0fdb04ad-e08a-4b27-aa43-5f5ec0e7c92c","Type":"ContainerStarted","Data":"49b3ea256aa57501548d9413ee30b83976d1527daafc0f44ecba0aad3e2cec20"}
Sep 11 01:25:55.379334 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:55.379289    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7965-762bb7" podStartSLOduration=1.379274479 podStartE2EDuration="1.379274479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:25:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:25:55.377061139 +0000 UTC m=+1670.774253316" watchObservedRunningTime="2026-09-11 01:25:55.379274479 +0000 UTC m=+1670.776466652"
Sep 11 01:25:55.997548 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:55.997509    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7965-762bb7"]
Sep 11 01:25:56.002106 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:56.002075    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7965-762bb7"]
Sep 11 01:25:56.364251 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:25:56.364221    2574 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7965-762bb7" container="test-curl-7965-762bb7"
Sep 11 01:25:57.369588 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:57.369553    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7965-762bb7" podUID="0fdb04ad-e08a-4b27-aa43-5f5ec0e7c92c" containerName="test-curl-7965-762bb7" containerID="cri-o://13a2e531332576a9e0ce10a44e7413c289249def53fa5e76740d2ac9c2684ea1" gracePeriod=2
Sep 11 01:25:57.528348 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:57.528317    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55"]
Sep 11 01:25:57.555859 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:57.555834    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:25:58.585319 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:58.585287    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:25:59.558676 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:59.558651    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7965-762bb7"
Sep 11 01:25:59.600019 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:59.599997    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wpqfq\" (UniqueName: \"kubernetes.io/projected/0fdb04ad-e08a-4b27-aa43-5f5ec0e7c92c-kube-api-access-wpqfq\") pod \"0fdb04ad-e08a-4b27-aa43-5f5ec0e7c92c\" (UID: \"0fdb04ad-e08a-4b27-aa43-5f5ec0e7c92c\") "
Sep 11 01:25:59.602031 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:59.602010    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fdb04ad-e08a-4b27-aa43-5f5ec0e7c92c-kube-api-access-wpqfq" (OuterVolumeSpecName: "kube-api-access-wpqfq") pod "0fdb04ad-e08a-4b27-aa43-5f5ec0e7c92c" (UID: "0fdb04ad-e08a-4b27-aa43-5f5ec0e7c92c"). InnerVolumeSpecName "kube-api-access-wpqfq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:25:59.701408 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:25:59.701384    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpqfq\" (UniqueName: \"kubernetes.io/projected/0fdb04ad-e08a-4b27-aa43-5f5ec0e7c92c-kube-api-access-wpqfq\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:26:00.386363 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:00.386326    2574 generic.go:356] "Generic (PLEG): container finished" podID="0fdb04ad-e08a-4b27-aa43-5f5ec0e7c92c" containerID="13a2e531332576a9e0ce10a44e7413c289249def53fa5e76740d2ac9c2684ea1" exitCode=137
Sep 11 01:26:00.386576 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:00.386378    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7965-762bb7"
Sep 11 01:26:00.386576 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:00.386429    2574 scope.go:117] "RemoveContainer" containerID="13a2e531332576a9e0ce10a44e7413c289249def53fa5e76740d2ac9c2684ea1"
Sep 11 01:26:00.395567 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:00.395549    2574 scope.go:117] "RemoveContainer" containerID="13a2e531332576a9e0ce10a44e7413c289249def53fa5e76740d2ac9c2684ea1"
Sep 11 01:26:00.395855 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:26:00.395837    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13a2e531332576a9e0ce10a44e7413c289249def53fa5e76740d2ac9c2684ea1\": container with ID starting with 13a2e531332576a9e0ce10a44e7413c289249def53fa5e76740d2ac9c2684ea1 not found: ID does not exist" containerID="13a2e531332576a9e0ce10a44e7413c289249def53fa5e76740d2ac9c2684ea1"
Sep 11 01:26:00.395924 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:00.395863    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13a2e531332576a9e0ce10a44e7413c289249def53fa5e76740d2ac9c2684ea1"} err="failed to get container status \"13a2e531332576a9e0ce10a44e7413c289249def53fa5e76740d2ac9c2684ea1\": rpc error: code = NotFound desc = could not find container \"13a2e531332576a9e0ce10a44e7413c289249def53fa5e76740d2ac9c2684ea1\": container with ID starting with 13a2e531332576a9e0ce10a44e7413c289249def53fa5e76740d2ac9c2684ea1 not found: ID does not exist"
Sep 11 01:26:00.396184 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:26:00.396162    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7965-762bb7\" is forbidden: User \"system:node:ip-10-0-139-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-235.ec2.internal' and this object" podUID="0fdb04ad-e08a-4b27-aa43-5f5ec0e7c92c" pod="opendatahub/test-curl-7965-762bb7"
Sep 11 01:26:00.983438 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:00.983406    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:26:01.006331 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.006297    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-be588a-65d9794cf-9b2d7"]
Sep 11 01:26:01.006631 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.006601    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-be588a-65d9794cf-9b2d7" podUID="158ac01d-720e-4d0e-9d99-26fa0cf9f086" containerName="payload-processing" containerID="cri-o://5c129acc1571be4272d70ea656545f181054f6ac555483219452f0f2b56abd46" gracePeriod=30
Sep 11 01:26:01.016729 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.016293    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-be588a-6cdffc5cf8-xnhx6"]
Sep 11 01:26:01.016729 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.016573    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-be588a-6cdffc5cf8-xnhx6" podUID="b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e" containerName="payload-pre-processing" containerID="cri-o://c9ea480ee2db714dc5176306772ef9a9ce05c69d7bce9033fca1eadab086cefc" gracePeriod=30
Sep 11 01:26:01.027505 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:26:01.027467    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7965-762bb7\" is forbidden: User \"system:node:ip-10-0-139-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-235.ec2.internal' and this object" podUID="0fdb04ad-e08a-4b27-aa43-5f5ec0e7c92c" pod="opendatahub/test-curl-7965-762bb7"
Sep 11 01:26:01.044018 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:26:01.043982    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7965-762bb7\" is forbidden: User \"system:node:ip-10-0-139-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-235.ec2.internal' and this object" podUID="0fdb04ad-e08a-4b27-aa43-5f5ec0e7c92c" pod="opendatahub/test-curl-7965-762bb7"
Sep 11 01:26:01.047046 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.047026    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:26:01.100714 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.100684    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0fdb04ad-e08a-4b27-aa43-5f5ec0e7c92c" path="/var/lib/kubelet/pods/0fdb04ad-e08a-4b27-aa43-5f5ec0e7c92c/volumes"
Sep 11 01:26:01.112035 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.112013    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:26:01.176723 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.176630    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-be588a-fvq92"]
Sep 11 01:26:01.179304 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.178817    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:26:01.179935 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.179883    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-be588a-fvq92"]
Sep 11 01:26:01.235648 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.235580    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:26:01.284016 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.283971    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-be588a-65d9794cf-9b2d7"
Sep 11 01:26:01.286695 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.286676    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-be588a-6cdffc5cf8-xnhx6"
Sep 11 01:26:01.317049 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.317019    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/158ac01d-720e-4d0e-9d99-26fa0cf9f086-plugins-config-volume\") pod \"158ac01d-720e-4d0e-9d99-26fa0cf9f086\" (UID: \"158ac01d-720e-4d0e-9d99-26fa0cf9f086\") "
Sep 11 01:26:01.317169 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.317108    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhpv6\" (UniqueName: \"kubernetes.io/projected/158ac01d-720e-4d0e-9d99-26fa0cf9f086-kube-api-access-vhpv6\") pod \"158ac01d-720e-4d0e-9d99-26fa0cf9f086\" (UID: \"158ac01d-720e-4d0e-9d99-26fa0cf9f086\") "
Sep 11 01:26:01.317169 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.317156    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m9slh\" (UniqueName: \"kubernetes.io/projected/b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e-kube-api-access-m9slh\") pod \"b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e\" (UID: \"b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e\") "
Sep 11 01:26:01.317290 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.317188    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e-plugins-config-volume\") pod \"b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e\" (UID: \"b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e\") "
Sep 11 01:26:01.317422 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.317397    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/158ac01d-720e-4d0e-9d99-26fa0cf9f086-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "158ac01d-720e-4d0e-9d99-26fa0cf9f086" (UID: "158ac01d-720e-4d0e-9d99-26fa0cf9f086"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:26:01.317685 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.317646    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e" (UID: "b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:26:01.318068 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.318047    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/158ac01d-720e-4d0e-9d99-26fa0cf9f086-plugins-config-volume\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:26:01.318192 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.318168    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e-plugins-config-volume\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:26:01.319490 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.319467    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/158ac01d-720e-4d0e-9d99-26fa0cf9f086-kube-api-access-vhpv6" (OuterVolumeSpecName: "kube-api-access-vhpv6") pod "158ac01d-720e-4d0e-9d99-26fa0cf9f086" (UID: "158ac01d-720e-4d0e-9d99-26fa0cf9f086"). InnerVolumeSpecName "kube-api-access-vhpv6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:26:01.319490 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.319473    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e-kube-api-access-m9slh" (OuterVolumeSpecName: "kube-api-access-m9slh") pod "b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e" (UID: "b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e"). InnerVolumeSpecName "kube-api-access-m9slh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:26:01.392297 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.392262    2574 generic.go:356] "Generic (PLEG): container finished" podID="b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e" containerID="c9ea480ee2db714dc5176306772ef9a9ce05c69d7bce9033fca1eadab086cefc" exitCode=0
Sep 11 01:26:01.392446 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.392311    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-be588a-6cdffc5cf8-xnhx6"
Sep 11 01:26:01.392446 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.392345    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-be588a-6cdffc5cf8-xnhx6" event={"ID":"b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e","Type":"ContainerDied","Data":"c9ea480ee2db714dc5176306772ef9a9ce05c69d7bce9033fca1eadab086cefc"}
Sep 11 01:26:01.392446 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.392398    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-be588a-6cdffc5cf8-xnhx6" event={"ID":"b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e","Type":"ContainerDied","Data":"30b674b323d2ca3e7e191e3eae37cb2d485fe70234523de0acf163f9a3057885"}
Sep 11 01:26:01.392446 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.392422    2574 scope.go:117] "RemoveContainer" containerID="c9ea480ee2db714dc5176306772ef9a9ce05c69d7bce9033fca1eadab086cefc"
Sep 11 01:26:01.394699 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.394674    2574 generic.go:356] "Generic (PLEG): container finished" podID="158ac01d-720e-4d0e-9d99-26fa0cf9f086" containerID="5c129acc1571be4272d70ea656545f181054f6ac555483219452f0f2b56abd46" exitCode=0
Sep 11 01:26:01.394808 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.394718    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-be588a-65d9794cf-9b2d7"
Sep 11 01:26:01.394808 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.394753    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-be588a-65d9794cf-9b2d7" event={"ID":"158ac01d-720e-4d0e-9d99-26fa0cf9f086","Type":"ContainerDied","Data":"5c129acc1571be4272d70ea656545f181054f6ac555483219452f0f2b56abd46"}
Sep 11 01:26:01.394808 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.394803    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-be588a-65d9794cf-9b2d7" event={"ID":"158ac01d-720e-4d0e-9d99-26fa0cf9f086","Type":"ContainerDied","Data":"c885741cb7617bdf830501f2269b4eb8f72e674ccae886a92029c5f8b8f7ce2c"}
Sep 11 01:26:01.401357 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.401341    2574 scope.go:117] "RemoveContainer" containerID="c9ea480ee2db714dc5176306772ef9a9ce05c69d7bce9033fca1eadab086cefc"
Sep 11 01:26:01.401658 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:26:01.401634    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9ea480ee2db714dc5176306772ef9a9ce05c69d7bce9033fca1eadab086cefc\": container with ID starting with c9ea480ee2db714dc5176306772ef9a9ce05c69d7bce9033fca1eadab086cefc not found: ID does not exist" containerID="c9ea480ee2db714dc5176306772ef9a9ce05c69d7bce9033fca1eadab086cefc"
Sep 11 01:26:01.401729 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.401671    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9ea480ee2db714dc5176306772ef9a9ce05c69d7bce9033fca1eadab086cefc"} err="failed to get container status \"c9ea480ee2db714dc5176306772ef9a9ce05c69d7bce9033fca1eadab086cefc\": rpc error: code = NotFound desc = could not find container \"c9ea480ee2db714dc5176306772ef9a9ce05c69d7bce9033fca1eadab086cefc\": container with ID starting with c9ea480ee2db714dc5176306772ef9a9ce05c69d7bce9033fca1eadab086cefc not found: ID does not exist"
Sep 11 01:26:01.401729 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.401694    2574 scope.go:117] "RemoveContainer" containerID="5c129acc1571be4272d70ea656545f181054f6ac555483219452f0f2b56abd46"
Sep 11 01:26:01.409583 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.409564    2574 scope.go:117] "RemoveContainer" containerID="5c129acc1571be4272d70ea656545f181054f6ac555483219452f0f2b56abd46"
Sep 11 01:26:01.409873 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:26:01.409852    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c129acc1571be4272d70ea656545f181054f6ac555483219452f0f2b56abd46\": container with ID starting with 5c129acc1571be4272d70ea656545f181054f6ac555483219452f0f2b56abd46 not found: ID does not exist" containerID="5c129acc1571be4272d70ea656545f181054f6ac555483219452f0f2b56abd46"
Sep 11 01:26:01.409997 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.409881    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c129acc1571be4272d70ea656545f181054f6ac555483219452f0f2b56abd46"} err="failed to get container status \"5c129acc1571be4272d70ea656545f181054f6ac555483219452f0f2b56abd46\": rpc error: code = NotFound desc = could not find container \"5c129acc1571be4272d70ea656545f181054f6ac555483219452f0f2b56abd46\": container with ID starting with 5c129acc1571be4272d70ea656545f181054f6ac555483219452f0f2b56abd46 not found: ID does not exist"
Sep 11 01:26:01.414740 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.414719    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-be588a-6cdffc5cf8-xnhx6"]
Sep 11 01:26:01.418084 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.418064    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-be588a-6cdffc5cf8-xnhx6"]
Sep 11 01:26:01.418860 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.418845    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhpv6\" (UniqueName: \"kubernetes.io/projected/158ac01d-720e-4d0e-9d99-26fa0cf9f086-kube-api-access-vhpv6\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:26:01.418933 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.418862    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9slh\" (UniqueName: \"kubernetes.io/projected/b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e-kube-api-access-m9slh\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:26:01.427363 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.427343    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-be588a-65d9794cf-9b2d7"]
Sep 11 01:26:01.432445 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.432421    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-be588a-65d9794cf-9b2d7"]
Sep 11 01:26:01.613274 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:01.613181    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:26:03.100744 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:03.100698    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="158ac01d-720e-4d0e-9d99-26fa0cf9f086" path="/var/lib/kubelet/pods/158ac01d-720e-4d0e-9d99-26fa0cf9f086/volumes"
Sep 11 01:26:03.101267 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:03.101224    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e" path="/var/lib/kubelet/pods/b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e/volumes"
Sep 11 01:26:03.101577 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:03.101557    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fae00d0e-b85d-4666-b540-e823be346d07" path="/var/lib/kubelet/pods/fae00d0e-b85d-4666-b540-e823be346d07/volumes"
Sep 11 01:26:03.887519 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:03.887487    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml"]
Sep 11 01:26:03.887945 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:03.887927    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="158ac01d-720e-4d0e-9d99-26fa0cf9f086" containerName="payload-processing"
Sep 11 01:26:03.888040 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:03.887946    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="158ac01d-720e-4d0e-9d99-26fa0cf9f086" containerName="payload-processing"
Sep 11 01:26:03.888040 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:03.887968    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e" containerName="payload-pre-processing"
Sep 11 01:26:03.888040 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:03.887976    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e" containerName="payload-pre-processing"
Sep 11 01:26:03.888040 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:03.887992    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fdb04ad-e08a-4b27-aa43-5f5ec0e7c92c" containerName="test-curl-7965-762bb7"
Sep 11 01:26:03.888040 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:03.888001    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fdb04ad-e08a-4b27-aa43-5f5ec0e7c92c" containerName="test-curl-7965-762bb7"
Sep 11 01:26:03.888284 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:03.888117    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="0fdb04ad-e08a-4b27-aa43-5f5ec0e7c92c" containerName="test-curl-7965-762bb7"
Sep 11 01:26:03.888284 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:03.888129    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b55bf1ad-c538-4be5-87c4-bd32ed6d9c8e" containerName="payload-pre-processing"
Sep 11 01:26:03.888284 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:03.888142    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="158ac01d-720e-4d0e-9d99-26fa0cf9f086" containerName="payload-processing"
Sep 11 01:26:03.892644 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:03.892623    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml"
Sep 11 01:26:03.894757 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:03.894738    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-d6e26b\"/\"sub-test-model-d6e26b-kserve-self-signed-certs\""
Sep 11 01:26:03.899176 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:03.899102    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml"]
Sep 11 01:26:03.905489 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:03.905464    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:26:03.940537 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:03.940514    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d6b3202-91cc-4444-92c5-72cec717ee2d-home\") pod \"sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml\" (UID: \"0d6b3202-91cc-4444-92c5-72cec717ee2d\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml"
Sep 11 01:26:03.940671 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:03.940572    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d6b3202-91cc-4444-92c5-72cec717ee2d-model-cache\") pod \"sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml\" (UID: \"0d6b3202-91cc-4444-92c5-72cec717ee2d\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml"
Sep 11 01:26:03.940671 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:03.940616    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d6b3202-91cc-4444-92c5-72cec717ee2d-dshm\") pod \"sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml\" (UID: \"0d6b3202-91cc-4444-92c5-72cec717ee2d\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml"
Sep 11 01:26:03.940671 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:03.940641    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d6b3202-91cc-4444-92c5-72cec717ee2d-tmp-dir\") pod \"sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml\" (UID: \"0d6b3202-91cc-4444-92c5-72cec717ee2d\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml"
Sep 11 01:26:03.940796 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:03.940685    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d6b3202-91cc-4444-92c5-72cec717ee2d-tls-certs\") pod \"sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml\" (UID: \"0d6b3202-91cc-4444-92c5-72cec717ee2d\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml"
Sep 11 01:26:03.940796 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:03.940729    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqlbp\" (UniqueName: \"kubernetes.io/projected/0d6b3202-91cc-4444-92c5-72cec717ee2d-kube-api-access-zqlbp\") pod \"sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml\" (UID: \"0d6b3202-91cc-4444-92c5-72cec717ee2d\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml"
Sep 11 01:26:04.041581 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:04.041545    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d6b3202-91cc-4444-92c5-72cec717ee2d-dshm\") pod \"sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml\" (UID: \"0d6b3202-91cc-4444-92c5-72cec717ee2d\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml"
Sep 11 01:26:04.041748 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:04.041591    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d6b3202-91cc-4444-92c5-72cec717ee2d-tmp-dir\") pod \"sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml\" (UID: \"0d6b3202-91cc-4444-92c5-72cec717ee2d\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml"
Sep 11 01:26:04.041748 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:04.041612    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d6b3202-91cc-4444-92c5-72cec717ee2d-tls-certs\") pod \"sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml\" (UID: \"0d6b3202-91cc-4444-92c5-72cec717ee2d\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml"
Sep 11 01:26:04.041748 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:04.041661    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqlbp\" (UniqueName: \"kubernetes.io/projected/0d6b3202-91cc-4444-92c5-72cec717ee2d-kube-api-access-zqlbp\") pod \"sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml\" (UID: \"0d6b3202-91cc-4444-92c5-72cec717ee2d\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml"
Sep 11 01:26:04.041748 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:04.041703    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d6b3202-91cc-4444-92c5-72cec717ee2d-home\") pod \"sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml\" (UID: \"0d6b3202-91cc-4444-92c5-72cec717ee2d\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml"
Sep 11 01:26:04.042028 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:04.041752    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d6b3202-91cc-4444-92c5-72cec717ee2d-model-cache\") pod \"sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml\" (UID: \"0d6b3202-91cc-4444-92c5-72cec717ee2d\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml"
Sep 11 01:26:04.042192 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:04.042167    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d6b3202-91cc-4444-92c5-72cec717ee2d-home\") pod \"sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml\" (UID: \"0d6b3202-91cc-4444-92c5-72cec717ee2d\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml"
Sep 11 01:26:04.042245 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:04.042186    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d6b3202-91cc-4444-92c5-72cec717ee2d-model-cache\") pod \"sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml\" (UID: \"0d6b3202-91cc-4444-92c5-72cec717ee2d\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml"
Sep 11 01:26:04.042296 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:04.042258    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d6b3202-91cc-4444-92c5-72cec717ee2d-tmp-dir\") pod \"sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml\" (UID: \"0d6b3202-91cc-4444-92c5-72cec717ee2d\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml"
Sep 11 01:26:04.044019 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:04.043995    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d6b3202-91cc-4444-92c5-72cec717ee2d-dshm\") pod \"sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml\" (UID: \"0d6b3202-91cc-4444-92c5-72cec717ee2d\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml"
Sep 11 01:26:04.044877 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:04.044854    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d6b3202-91cc-4444-92c5-72cec717ee2d-tls-certs\") pod \"sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml\" (UID: \"0d6b3202-91cc-4444-92c5-72cec717ee2d\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml"
Sep 11 01:26:04.049835 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:04.049806    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqlbp\" (UniqueName: \"kubernetes.io/projected/0d6b3202-91cc-4444-92c5-72cec717ee2d-kube-api-access-zqlbp\") pod \"sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml\" (UID: \"0d6b3202-91cc-4444-92c5-72cec717ee2d\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml"
Sep 11 01:26:04.203059 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:04.203026    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml"
Sep 11 01:26:04.473505 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:04.473481    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml"]
Sep 11 01:26:04.476098 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:26:04.476074    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0d6b3202_91cc_4444_92c5_72cec717ee2d.slice/crio-5e72711974c9db08c0311eec36bb39cace8de1486fb728a1833634eb3d0f324f WatchSource:0}: Error finding container 5e72711974c9db08c0311eec36bb39cace8de1486fb728a1833634eb3d0f324f: Status 404 returned error can't find the container with id 5e72711974c9db08c0311eec36bb39cace8de1486fb728a1833634eb3d0f324f
Sep 11 01:26:05.420232 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:05.420194    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml" event={"ID":"0d6b3202-91cc-4444-92c5-72cec717ee2d","Type":"ContainerStarted","Data":"d0e2a1da461c3f7b3f9c41d227371de0c3e31fe90c85c750ef7f860a7b083147"}
Sep 11 01:26:05.420232 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:05.420236    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml" event={"ID":"0d6b3202-91cc-4444-92c5-72cec717ee2d","Type":"ContainerStarted","Data":"5e72711974c9db08c0311eec36bb39cace8de1486fb728a1833634eb3d0f324f"}
Sep 11 01:26:05.436746 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:05.436699    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml" podStartSLOduration=2.436685587 podStartE2EDuration="2.436685587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:26:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:26:05.435000955 +0000 UTC m=+1680.832193127" watchObservedRunningTime="2026-09-11 01:26:05.436685587 +0000 UTC m=+1680.833877759"
Sep 11 01:26:07.357631 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:07.357596    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:26:07.456614 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:07.456569    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:26:07.651900 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:07.651809    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:26:08.091705 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:08.091667    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-76bb56-p6zs2"]
Sep 11 01:26:08.095057 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:08.095036    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-76bb56-p6zs2"
Sep 11 01:26:08.097405 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:08.097385    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 01:26:08.097517 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:08.097386    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q6gbn\""
Sep 11 01:26:08.102853 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:08.102831    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-76bb56-p6zs2"]
Sep 11 01:26:08.179649 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:08.179613    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44bf09fb-dfa5-4cc2-85f9-e1d99d88304e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-76bb56-p6zs2\" (UID: \"44bf09fb-dfa5-4cc2-85f9-e1d99d88304e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-76bb56-p6zs2"
Sep 11 01:26:08.280362 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:08.280326    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44bf09fb-dfa5-4cc2-85f9-e1d99d88304e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-76bb56-p6zs2\" (UID: \"44bf09fb-dfa5-4cc2-85f9-e1d99d88304e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-76bb56-p6zs2"
Sep 11 01:26:08.281028 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:08.281007    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44bf09fb-dfa5-4cc2-85f9-e1d99d88304e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-76bb56-p6zs2\" (UID: \"44bf09fb-dfa5-4cc2-85f9-e1d99d88304e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-76bb56-p6zs2"
Sep 11 01:26:08.343033 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:08.342950    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:26:08.405763 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:08.405730    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-76bb56-p6zs2"
Sep 11 01:26:08.676301 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:08.676275    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-76bb56-p6zs2"]
Sep 11 01:26:08.678733 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:26:08.678703    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44bf09fb_dfa5_4cc2_85f9_e1d99d88304e.slice/crio-0316ea87e56f121f4cecabbe6e38f61d833b4ac4b9b2a4d250e89808a6f84e34 WatchSource:0}: Error finding container 0316ea87e56f121f4cecabbe6e38f61d833b4ac4b9b2a4d250e89808a6f84e34: Status 404 returned error can't find the container with id 0316ea87e56f121f4cecabbe6e38f61d833b4ac4b9b2a4d250e89808a6f84e34
Sep 11 01:26:09.441996 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:09.441961    2574 generic.go:356] "Generic (PLEG): container finished" podID="44bf09fb-dfa5-4cc2-85f9-e1d99d88304e" containerID="232af026be7c68d431c5b653d299c7b9251a721905e32f11a26ee159845b1912" exitCode=0
Sep 11 01:26:09.442388 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:09.442007    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-76bb56-p6zs2" event={"ID":"44bf09fb-dfa5-4cc2-85f9-e1d99d88304e","Type":"ContainerDied","Data":"232af026be7c68d431c5b653d299c7b9251a721905e32f11a26ee159845b1912"}
Sep 11 01:26:09.442388 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:09.442049    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-76bb56-p6zs2" event={"ID":"44bf09fb-dfa5-4cc2-85f9-e1d99d88304e","Type":"ContainerStarted","Data":"0316ea87e56f121f4cecabbe6e38f61d833b4ac4b9b2a4d250e89808a6f84e34"}
Sep 11 01:26:10.578291 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:10.578265    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-76bb56-p6zs2"
Sep 11 01:26:10.701053 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:10.700972    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44bf09fb-dfa5-4cc2-85f9-e1d99d88304e-maas-api-service-ca\") pod \"44bf09fb-dfa5-4cc2-85f9-e1d99d88304e\" (UID: \"44bf09fb-dfa5-4cc2-85f9-e1d99d88304e\") "
Sep 11 01:26:10.701442 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:10.701411    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44bf09fb-dfa5-4cc2-85f9-e1d99d88304e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "44bf09fb-dfa5-4cc2-85f9-e1d99d88304e" (UID: "44bf09fb-dfa5-4cc2-85f9-e1d99d88304e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:26:10.802338 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:10.802304    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44bf09fb-dfa5-4cc2-85f9-e1d99d88304e-maas-api-service-ca\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:26:11.454793 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:11.454755    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-76bb56-p6zs2" event={"ID":"44bf09fb-dfa5-4cc2-85f9-e1d99d88304e","Type":"ContainerDied","Data":"0316ea87e56f121f4cecabbe6e38f61d833b4ac4b9b2a4d250e89808a6f84e34"}
Sep 11 01:26:11.454793 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:11.454780    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-76bb56-p6zs2"
Sep 11 01:26:11.455023 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:11.454803    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0316ea87e56f121f4cecabbe6e38f61d833b4ac4b9b2a4d250e89808a6f84e34"
Sep 11 01:26:12.548636 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:12.548598    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55" podUID="b6f32ac1-8184-42b3-b120-0d6e5c4fccc0" containerName="main" containerID="cri-o://49adcb220d11a1d3b29735a7621137fdaf1b416f131a2197382a1621db8f1737" gracePeriod=45
Sep 11 01:26:12.813417 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:12.813360    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55"
Sep 11 01:26:12.920775 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:12.920739    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-tmp-dir\") pod \"b6f32ac1-8184-42b3-b120-0d6e5c4fccc0\" (UID: \"b6f32ac1-8184-42b3-b120-0d6e5c4fccc0\") "
Sep 11 01:26:12.920966 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:12.920823    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9c7v7\" (UniqueName: \"kubernetes.io/projected/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-kube-api-access-9c7v7\") pod \"b6f32ac1-8184-42b3-b120-0d6e5c4fccc0\" (UID: \"b6f32ac1-8184-42b3-b120-0d6e5c4fccc0\") "
Sep 11 01:26:12.920966 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:12.920948    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-dshm\") pod \"b6f32ac1-8184-42b3-b120-0d6e5c4fccc0\" (UID: \"b6f32ac1-8184-42b3-b120-0d6e5c4fccc0\") "
Sep 11 01:26:12.921068 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:12.920989    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-model-cache\") pod \"b6f32ac1-8184-42b3-b120-0d6e5c4fccc0\" (UID: \"b6f32ac1-8184-42b3-b120-0d6e5c4fccc0\") "
Sep 11 01:26:12.921068 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:12.921020    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-tls-certs\") pod \"b6f32ac1-8184-42b3-b120-0d6e5c4fccc0\" (UID: \"b6f32ac1-8184-42b3-b120-0d6e5c4fccc0\") "
Sep 11 01:26:12.921068 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:12.921046    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-home\") pod \"b6f32ac1-8184-42b3-b120-0d6e5c4fccc0\" (UID: \"b6f32ac1-8184-42b3-b120-0d6e5c4fccc0\") "
Sep 11 01:26:12.921068 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:12.921046    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b6f32ac1-8184-42b3-b120-0d6e5c4fccc0" (UID: "b6f32ac1-8184-42b3-b120-0d6e5c4fccc0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:26:12.921283 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:12.921262    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-model-cache" (OuterVolumeSpecName: "model-cache") pod "b6f32ac1-8184-42b3-b120-0d6e5c4fccc0" (UID: "b6f32ac1-8184-42b3-b120-0d6e5c4fccc0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:26:12.921377 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:12.921347    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-home" (OuterVolumeSpecName: "home") pod "b6f32ac1-8184-42b3-b120-0d6e5c4fccc0" (UID: "b6f32ac1-8184-42b3-b120-0d6e5c4fccc0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:26:12.921500 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:12.921403    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-model-cache\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:26:12.921500 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:12.921419    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-tmp-dir\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:26:12.923263 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:12.923235    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b6f32ac1-8184-42b3-b120-0d6e5c4fccc0" (UID: "b6f32ac1-8184-42b3-b120-0d6e5c4fccc0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:26:12.923369 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:12.923265    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-kube-api-access-9c7v7" (OuterVolumeSpecName: "kube-api-access-9c7v7") pod "b6f32ac1-8184-42b3-b120-0d6e5c4fccc0" (UID: "b6f32ac1-8184-42b3-b120-0d6e5c4fccc0"). InnerVolumeSpecName "kube-api-access-9c7v7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:26:12.923430 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:12.923381    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-dshm" (OuterVolumeSpecName: "dshm") pod "b6f32ac1-8184-42b3-b120-0d6e5c4fccc0" (UID: "b6f32ac1-8184-42b3-b120-0d6e5c4fccc0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:26:13.021834 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:13.021799    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-dshm\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:26:13.021834 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:13.021826    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-tls-certs\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:26:13.021834 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:13.021834    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-home\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:26:13.022085 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:13.021842    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9c7v7\" (UniqueName: \"kubernetes.io/projected/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0-kube-api-access-9c7v7\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:26:13.466276 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:13.466241    2574 generic.go:356] "Generic (PLEG): container finished" podID="b6f32ac1-8184-42b3-b120-0d6e5c4fccc0" containerID="49adcb220d11a1d3b29735a7621137fdaf1b416f131a2197382a1621db8f1737" exitCode=0
Sep 11 01:26:13.466457 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:13.466285    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55"
Sep 11 01:26:13.466457 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:13.466323    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55" event={"ID":"b6f32ac1-8184-42b3-b120-0d6e5c4fccc0","Type":"ContainerDied","Data":"49adcb220d11a1d3b29735a7621137fdaf1b416f131a2197382a1621db8f1737"}
Sep 11 01:26:13.466457 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:13.466369    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55" event={"ID":"b6f32ac1-8184-42b3-b120-0d6e5c4fccc0","Type":"ContainerDied","Data":"a4b4f2b4efa3a52983d153df8f1b14c45a2a1ed5114623b195b16e438c76752b"}
Sep 11 01:26:13.466457 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:13.466392    2574 scope.go:117] "RemoveContainer" containerID="49adcb220d11a1d3b29735a7621137fdaf1b416f131a2197382a1621db8f1737"
Sep 11 01:26:13.475388 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:13.475367    2574 scope.go:117] "RemoveContainer" containerID="49adcb220d11a1d3b29735a7621137fdaf1b416f131a2197382a1621db8f1737"
Sep 11 01:26:13.475641 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:26:13.475625    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49adcb220d11a1d3b29735a7621137fdaf1b416f131a2197382a1621db8f1737\": container with ID starting with 49adcb220d11a1d3b29735a7621137fdaf1b416f131a2197382a1621db8f1737 not found: ID does not exist" containerID="49adcb220d11a1d3b29735a7621137fdaf1b416f131a2197382a1621db8f1737"
Sep 11 01:26:13.475692 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:13.475654    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49adcb220d11a1d3b29735a7621137fdaf1b416f131a2197382a1621db8f1737"} err="failed to get container status \"49adcb220d11a1d3b29735a7621137fdaf1b416f131a2197382a1621db8f1737\": rpc error: code = NotFound desc = could not find container \"49adcb220d11a1d3b29735a7621137fdaf1b416f131a2197382a1621db8f1737\": container with ID starting with 49adcb220d11a1d3b29735a7621137fdaf1b416f131a2197382a1621db8f1737 not found: ID does not exist"
Sep 11 01:26:13.481092 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:13.481071    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55"]
Sep 11 01:26:13.486443 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:13.486414    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-d6e26b/auth-test-model-d6e26b-kserve-865b575cbc-wjr55"]
Sep 11 01:26:14.203299 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:14.203262    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml"
Sep 11 01:26:14.203732 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:14.203317    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml"
Sep 11 01:26:14.216236 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:14.216210    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml"
Sep 11 01:26:14.483829 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:14.483741    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml"
Sep 11 01:26:15.100152 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:15.100115    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6f32ac1-8184-42b3-b120-0d6e5c4fccc0" path="/var/lib/kubelet/pods/b6f32ac1-8184-42b3-b120-0d6e5c4fccc0/volumes"
Sep 11 01:26:18.643741 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:18.643709    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:26:18.711308 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:18.711270    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:26:18.808260 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:18.808228    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:26:18.831342 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:18.831300    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-76bb56-p6zs2"]
Sep 11 01:26:18.833019 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:18.832994    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-76bb56-p6zs2"]
Sep 11 01:26:18.868610 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:18.868576    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:26:19.100166 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:19.100129    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44bf09fb-dfa5-4cc2-85f9-e1d99d88304e" path="/var/lib/kubelet/pods/44bf09fb-dfa5-4cc2-85f9-e1d99d88304e/volumes"
Sep 11 01:26:19.216462 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:19.216430    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:26:25.383323 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:25.383291    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:26:25.482889 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:25.482847    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:26:25.699396 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:25.699365    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:26:26.097724 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:26.097650    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4f6824-kkxwx"]
Sep 11 01:26:26.098021 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:26.098008    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44bf09fb-dfa5-4cc2-85f9-e1d99d88304e" containerName="revoke-keys"
Sep 11 01:26:26.098062 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:26.098023    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="44bf09fb-dfa5-4cc2-85f9-e1d99d88304e" containerName="revoke-keys"
Sep 11 01:26:26.098062 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:26.098040    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6f32ac1-8184-42b3-b120-0d6e5c4fccc0" containerName="main"
Sep 11 01:26:26.098062 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:26.098045    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6f32ac1-8184-42b3-b120-0d6e5c4fccc0" containerName="main"
Sep 11 01:26:26.098172 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:26.098105    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="44bf09fb-dfa5-4cc2-85f9-e1d99d88304e" containerName="revoke-keys"
Sep 11 01:26:26.098172 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:26.098114    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6f32ac1-8184-42b3-b120-0d6e5c4fccc0" containerName="main"
Sep 11 01:26:26.102672 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:26.102655    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4f6824-kkxwx"
Sep 11 01:26:26.105048 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:26.105025    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 01:26:26.105377 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:26.105099    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q6gbn\""
Sep 11 01:26:26.108305 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:26.108283    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4f6824-kkxwx"]
Sep 11 01:26:26.203057 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:26.203023    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:26:26.244863 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:26.244835    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d9849d6-feba-4899-b0c3-2af1e252bba6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-4f6824-kkxwx\" (UID: \"5d9849d6-feba-4899-b0c3-2af1e252bba6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4f6824-kkxwx"
Sep 11 01:26:26.346081 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:26.346019    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d9849d6-feba-4899-b0c3-2af1e252bba6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-4f6824-kkxwx\" (UID: \"5d9849d6-feba-4899-b0c3-2af1e252bba6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4f6824-kkxwx"
Sep 11 01:26:26.346608 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:26.346590    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d9849d6-feba-4899-b0c3-2af1e252bba6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-4f6824-kkxwx\" (UID: \"5d9849d6-feba-4899-b0c3-2af1e252bba6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4f6824-kkxwx"
Sep 11 01:26:26.414153 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:26.414133    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4f6824-kkxwx"
Sep 11 01:26:26.688673 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:26.688639    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4f6824-kkxwx"]
Sep 11 01:26:26.691076 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:26:26.691047    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d9849d6_feba_4899_b0c3_2af1e252bba6.slice/crio-46e07a7476f76d73d0059a61ea17cb4da130c616fd8e55ddd75b5aed8af72561 WatchSource:0}: Error finding container 46e07a7476f76d73d0059a61ea17cb4da130c616fd8e55ddd75b5aed8af72561: Status 404 returned error can't find the container with id 46e07a7476f76d73d0059a61ea17cb4da130c616fd8e55ddd75b5aed8af72561
Sep 11 01:26:27.538009 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:27.537971    2574 generic.go:356] "Generic (PLEG): container finished" podID="5d9849d6-feba-4899-b0c3-2af1e252bba6" containerID="4d29509998f3b41f6b0e4da6f616b989b54f2ca988b0a1434658e2dce7595ee4" exitCode=0
Sep 11 01:26:27.538438 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:27.538063    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4f6824-kkxwx" event={"ID":"5d9849d6-feba-4899-b0c3-2af1e252bba6","Type":"ContainerDied","Data":"4d29509998f3b41f6b0e4da6f616b989b54f2ca988b0a1434658e2dce7595ee4"}
Sep 11 01:26:27.538438 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:27.538124    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4f6824-kkxwx" event={"ID":"5d9849d6-feba-4899-b0c3-2af1e252bba6","Type":"ContainerStarted","Data":"46e07a7476f76d73d0059a61ea17cb4da130c616fd8e55ddd75b5aed8af72561"}
Sep 11 01:26:28.673161 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:28.673135    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4f6824-kkxwx"
Sep 11 01:26:28.767224 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:28.767195    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d9849d6-feba-4899-b0c3-2af1e252bba6-maas-api-service-ca\") pod \"5d9849d6-feba-4899-b0c3-2af1e252bba6\" (UID: \"5d9849d6-feba-4899-b0c3-2af1e252bba6\") "
Sep 11 01:26:28.767562 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:28.767536    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d9849d6-feba-4899-b0c3-2af1e252bba6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5d9849d6-feba-4899-b0c3-2af1e252bba6" (UID: "5d9849d6-feba-4899-b0c3-2af1e252bba6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:26:28.868825 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:28.868762    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d9849d6-feba-4899-b0c3-2af1e252bba6-maas-api-service-ca\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:26:29.549373 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:29.549340    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4f6824-kkxwx" event={"ID":"5d9849d6-feba-4899-b0c3-2af1e252bba6","Type":"ContainerDied","Data":"46e07a7476f76d73d0059a61ea17cb4da130c616fd8e55ddd75b5aed8af72561"}
Sep 11 01:26:29.549373 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:29.549371    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4f6824-kkxwx"
Sep 11 01:26:29.549619 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:29.549382    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="46e07a7476f76d73d0059a61ea17cb4da130c616fd8e55ddd75b5aed8af72561"
Sep 11 01:26:36.616428 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:36.616391    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:26:36.636850 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:36.636815    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-4f6824-68859fc547-47gtd"]
Sep 11 01:26:36.637042 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:36.637019    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f6824-68859fc547-47gtd" podUID="6a26faec-cbdf-47c5-a3e0-f52dc9047999" containerName="payload-processing" containerID="cri-o://d6a1bd6c7478e632245b69963c181b2287af65cff28db097702d2b43ef4bd620" gracePeriod=30
Sep 11 01:26:36.641760 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:36.641723    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f6824-647f95d65d-kflvl"]
Sep 11 01:26:36.642016 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:36.641988    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f6824-647f95d65d-kflvl" podUID="19cda1a2-f6b5-4856-89a6-12fcb724ea95" containerName="payload-pre-processing" containerID="cri-o://55657a97eefa2aff970a87cccff3aba8292cc569f6e7b6669107daf6c84c7605" gracePeriod=30
Sep 11 01:26:36.678413 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:36.678385    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:26:36.741105 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:36.741074    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:26:36.791055 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:36.791012    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4f6824-kkxwx"]
Sep 11 01:26:36.794998 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:36.794970    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-4f6824-kkxwx"]
Sep 11 01:26:36.797619 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:36.797589    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:26:36.916332 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:36.916309    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f6824-647f95d65d-kflvl"
Sep 11 01:26:36.919487 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:36.919468    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f6824-68859fc547-47gtd"
Sep 11 01:26:36.936338 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:36.936308    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hpds\" (UniqueName: \"kubernetes.io/projected/6a26faec-cbdf-47c5-a3e0-f52dc9047999-kube-api-access-2hpds\") pod \"6a26faec-cbdf-47c5-a3e0-f52dc9047999\" (UID: \"6a26faec-cbdf-47c5-a3e0-f52dc9047999\") "
Sep 11 01:26:36.936450 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:36.936368    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a26faec-cbdf-47c5-a3e0-f52dc9047999-plugins-config-volume\") pod \"6a26faec-cbdf-47c5-a3e0-f52dc9047999\" (UID: \"6a26faec-cbdf-47c5-a3e0-f52dc9047999\") "
Sep 11 01:26:36.936509 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:36.936471    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2zlkj\" (UniqueName: \"kubernetes.io/projected/19cda1a2-f6b5-4856-89a6-12fcb724ea95-kube-api-access-2zlkj\") pod \"19cda1a2-f6b5-4856-89a6-12fcb724ea95\" (UID: \"19cda1a2-f6b5-4856-89a6-12fcb724ea95\") "
Sep 11 01:26:36.936509 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:36.936496    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19cda1a2-f6b5-4856-89a6-12fcb724ea95-plugins-config-volume\") pod \"19cda1a2-f6b5-4856-89a6-12fcb724ea95\" (UID: \"19cda1a2-f6b5-4856-89a6-12fcb724ea95\") "
Sep 11 01:26:36.936891 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:36.936826    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a26faec-cbdf-47c5-a3e0-f52dc9047999-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6a26faec-cbdf-47c5-a3e0-f52dc9047999" (UID: "6a26faec-cbdf-47c5-a3e0-f52dc9047999"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:26:36.937049 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:36.937027    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19cda1a2-f6b5-4856-89a6-12fcb724ea95-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "19cda1a2-f6b5-4856-89a6-12fcb724ea95" (UID: "19cda1a2-f6b5-4856-89a6-12fcb724ea95"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:26:36.938804 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:36.938770    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19cda1a2-f6b5-4856-89a6-12fcb724ea95-kube-api-access-2zlkj" (OuterVolumeSpecName: "kube-api-access-2zlkj") pod "19cda1a2-f6b5-4856-89a6-12fcb724ea95" (UID: "19cda1a2-f6b5-4856-89a6-12fcb724ea95"). InnerVolumeSpecName "kube-api-access-2zlkj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:26:36.939018 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:36.938994    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a26faec-cbdf-47c5-a3e0-f52dc9047999-kube-api-access-2hpds" (OuterVolumeSpecName: "kube-api-access-2hpds") pod "6a26faec-cbdf-47c5-a3e0-f52dc9047999" (UID: "6a26faec-cbdf-47c5-a3e0-f52dc9047999"). InnerVolumeSpecName "kube-api-access-2hpds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:26:37.037346 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:37.037320    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zlkj\" (UniqueName: \"kubernetes.io/projected/19cda1a2-f6b5-4856-89a6-12fcb724ea95-kube-api-access-2zlkj\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:26:37.037346 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:37.037342    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19cda1a2-f6b5-4856-89a6-12fcb724ea95-plugins-config-volume\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:26:37.037346 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:37.037352    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hpds\" (UniqueName: \"kubernetes.io/projected/6a26faec-cbdf-47c5-a3e0-f52dc9047999-kube-api-access-2hpds\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:26:37.037519 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:37.037362    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a26faec-cbdf-47c5-a3e0-f52dc9047999-plugins-config-volume\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:26:37.100049 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:37.100020    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d9849d6-feba-4899-b0c3-2af1e252bba6" path="/var/lib/kubelet/pods/5d9849d6-feba-4899-b0c3-2af1e252bba6/volumes"
Sep 11 01:26:37.213589 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:37.213556    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:26:37.593479 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:37.593398    2574 generic.go:356] "Generic (PLEG): container finished" podID="6a26faec-cbdf-47c5-a3e0-f52dc9047999" containerID="d6a1bd6c7478e632245b69963c181b2287af65cff28db097702d2b43ef4bd620" exitCode=0
Sep 11 01:26:37.593479 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:37.593448    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f6824-68859fc547-47gtd"
Sep 11 01:26:37.593709 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:37.593490    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f6824-68859fc547-47gtd" event={"ID":"6a26faec-cbdf-47c5-a3e0-f52dc9047999","Type":"ContainerDied","Data":"d6a1bd6c7478e632245b69963c181b2287af65cff28db097702d2b43ef4bd620"}
Sep 11 01:26:37.593709 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:37.593541    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f6824-68859fc547-47gtd" event={"ID":"6a26faec-cbdf-47c5-a3e0-f52dc9047999","Type":"ContainerDied","Data":"54efeb5d7cd4cec6d170d4632510ca31fe5d7b3b647daab72f86c89d71d73692"}
Sep 11 01:26:37.593709 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:37.593571    2574 scope.go:117] "RemoveContainer" containerID="d6a1bd6c7478e632245b69963c181b2287af65cff28db097702d2b43ef4bd620"
Sep 11 01:26:37.594865 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:37.594840    2574 generic.go:356] "Generic (PLEG): container finished" podID="19cda1a2-f6b5-4856-89a6-12fcb724ea95" containerID="55657a97eefa2aff970a87cccff3aba8292cc569f6e7b6669107daf6c84c7605" exitCode=0
Sep 11 01:26:37.594979 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:37.594886    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f6824-647f95d65d-kflvl"
Sep 11 01:26:37.594979 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:37.594952    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f6824-647f95d65d-kflvl" event={"ID":"19cda1a2-f6b5-4856-89a6-12fcb724ea95","Type":"ContainerDied","Data":"55657a97eefa2aff970a87cccff3aba8292cc569f6e7b6669107daf6c84c7605"}
Sep 11 01:26:37.595261 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:37.595009    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f6824-647f95d65d-kflvl" event={"ID":"19cda1a2-f6b5-4856-89a6-12fcb724ea95","Type":"ContainerDied","Data":"02e270e1959e48c09110eeedcf096da639bd6af271b9a223e384f46d712a1a76"}
Sep 11 01:26:37.602643 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:37.602629    2574 scope.go:117] "RemoveContainer" containerID="d6a1bd6c7478e632245b69963c181b2287af65cff28db097702d2b43ef4bd620"
Sep 11 01:26:37.602901 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:26:37.602886    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6a1bd6c7478e632245b69963c181b2287af65cff28db097702d2b43ef4bd620\": container with ID starting with d6a1bd6c7478e632245b69963c181b2287af65cff28db097702d2b43ef4bd620 not found: ID does not exist" containerID="d6a1bd6c7478e632245b69963c181b2287af65cff28db097702d2b43ef4bd620"
Sep 11 01:26:37.603000 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:37.602924    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6a1bd6c7478e632245b69963c181b2287af65cff28db097702d2b43ef4bd620"} err="failed to get container status \"d6a1bd6c7478e632245b69963c181b2287af65cff28db097702d2b43ef4bd620\": rpc error: code = NotFound desc = could not find container \"d6a1bd6c7478e632245b69963c181b2287af65cff28db097702d2b43ef4bd620\": container with ID starting with d6a1bd6c7478e632245b69963c181b2287af65cff28db097702d2b43ef4bd620 not found: ID does not exist"
Sep 11 01:26:37.603000 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:37.602940    2574 scope.go:117] "RemoveContainer" containerID="55657a97eefa2aff970a87cccff3aba8292cc569f6e7b6669107daf6c84c7605"
Sep 11 01:26:37.611072 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:37.611054    2574 scope.go:117] "RemoveContainer" containerID="55657a97eefa2aff970a87cccff3aba8292cc569f6e7b6669107daf6c84c7605"
Sep 11 01:26:37.611429 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:26:37.611387    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55657a97eefa2aff970a87cccff3aba8292cc569f6e7b6669107daf6c84c7605\": container with ID starting with 55657a97eefa2aff970a87cccff3aba8292cc569f6e7b6669107daf6c84c7605 not found: ID does not exist" containerID="55657a97eefa2aff970a87cccff3aba8292cc569f6e7b6669107daf6c84c7605"
Sep 11 01:26:37.611593 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:37.611569    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55657a97eefa2aff970a87cccff3aba8292cc569f6e7b6669107daf6c84c7605"} err="failed to get container status \"55657a97eefa2aff970a87cccff3aba8292cc569f6e7b6669107daf6c84c7605\": rpc error: code = NotFound desc = could not find container \"55657a97eefa2aff970a87cccff3aba8292cc569f6e7b6669107daf6c84c7605\": container with ID starting with 55657a97eefa2aff970a87cccff3aba8292cc569f6e7b6669107daf6c84c7605 not found: ID does not exist"
Sep 11 01:26:37.612311 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:37.612288    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-4f6824-68859fc547-47gtd"]
Sep 11 01:26:37.613930 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:37.613895    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-4f6824-68859fc547-47gtd"]
Sep 11 01:26:37.622957 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:37.622938    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f6824-647f95d65d-kflvl"]
Sep 11 01:26:37.627510 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:37.627488    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f6824-647f95d65d-kflvl"]
Sep 11 01:26:39.101021 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:39.100990    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19cda1a2-f6b5-4856-89a6-12fcb724ea95" path="/var/lib/kubelet/pods/19cda1a2-f6b5-4856-89a6-12fcb724ea95/volumes"
Sep 11 01:26:39.101388 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:39.101325    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a26faec-cbdf-47c5-a3e0-f52dc9047999" path="/var/lib/kubelet/pods/6a26faec-cbdf-47c5-a3e0-f52dc9047999/volumes"
Sep 11 01:26:42.957319 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:42.957287    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:26:43.016777 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:43.016750    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:26:43.261107 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:43.261027    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:26:54.415195 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:54.415155    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:26:56.703042 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:26:56.702998    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:27:05.535460 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:05.535432    2574 scope.go:117] "RemoveContainer" containerID="0590f870511ca435649b94cc54b4012b7d53af3c9c7387356d79a0520ad8fdaa"
Sep 11 01:27:25.032408 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:25.032374    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:27:51.952423 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:51.952380    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7965-371b4e"]
Sep 11 01:27:51.953073 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:51.952885    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a26faec-cbdf-47c5-a3e0-f52dc9047999" containerName="payload-processing"
Sep 11 01:27:51.953073 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:51.952925    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a26faec-cbdf-47c5-a3e0-f52dc9047999" containerName="payload-processing"
Sep 11 01:27:51.953073 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:51.952959    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19cda1a2-f6b5-4856-89a6-12fcb724ea95" containerName="payload-pre-processing"
Sep 11 01:27:51.953073 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:51.952968    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="19cda1a2-f6b5-4856-89a6-12fcb724ea95" containerName="payload-pre-processing"
Sep 11 01:27:51.953073 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:51.952993    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d9849d6-feba-4899-b0c3-2af1e252bba6" containerName="revoke-keys"
Sep 11 01:27:51.953073 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:51.953003    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d9849d6-feba-4899-b0c3-2af1e252bba6" containerName="revoke-keys"
Sep 11 01:27:51.953395 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:51.953104    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a26faec-cbdf-47c5-a3e0-f52dc9047999" containerName="payload-processing"
Sep 11 01:27:51.953395 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:51.953113    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d9849d6-feba-4899-b0c3-2af1e252bba6" containerName="revoke-keys"
Sep 11 01:27:51.953395 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:51.953125    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="19cda1a2-f6b5-4856-89a6-12fcb724ea95" containerName="payload-pre-processing"
Sep 11 01:27:51.956349 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:51.956327    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7965-371b4e"
Sep 11 01:27:51.958646 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:51.958625    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-7gdtv\""
Sep 11 01:27:51.961194 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:51.961167    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7965-371b4e"]
Sep 11 01:27:52.086844 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:52.086789    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kn8d7\" (UniqueName: \"kubernetes.io/projected/aa2bc96b-f4d2-4144-b615-fcfe9b34c859-kube-api-access-kn8d7\") pod \"test-curl-7965-371b4e\" (UID: \"aa2bc96b-f4d2-4144-b615-fcfe9b34c859\") " pod="opendatahub/test-curl-7965-371b4e"
Sep 11 01:27:52.187884 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:52.187849    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kn8d7\" (UniqueName: \"kubernetes.io/projected/aa2bc96b-f4d2-4144-b615-fcfe9b34c859-kube-api-access-kn8d7\") pod \"test-curl-7965-371b4e\" (UID: \"aa2bc96b-f4d2-4144-b615-fcfe9b34c859\") " pod="opendatahub/test-curl-7965-371b4e"
Sep 11 01:27:52.195555 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:52.195530    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kn8d7\" (UniqueName: \"kubernetes.io/projected/aa2bc96b-f4d2-4144-b615-fcfe9b34c859-kube-api-access-kn8d7\") pod \"test-curl-7965-371b4e\" (UID: \"aa2bc96b-f4d2-4144-b615-fcfe9b34c859\") " pod="opendatahub/test-curl-7965-371b4e"
Sep 11 01:27:52.267745 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:52.267655    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7965-371b4e"
Sep 11 01:27:52.536213 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:52.536188    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7965-371b4e"]
Sep 11 01:27:52.538398 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:27:52.538361    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaa2bc96b_f4d2_4144_b615_fcfe9b34c859.slice/crio-398db80914fee0831604b450968a293b1ef18e34274206c059bdfe0252e415cc WatchSource:0}: Error finding container 398db80914fee0831604b450968a293b1ef18e34274206c059bdfe0252e415cc: Status 404 returned error can't find the container with id 398db80914fee0831604b450968a293b1ef18e34274206c059bdfe0252e415cc
Sep 11 01:27:52.971259 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:52.971221    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7965-371b4e" event={"ID":"aa2bc96b-f4d2-4144-b615-fcfe9b34c859","Type":"ContainerStarted","Data":"dadc20f3ec30539149db1d1fcaceca0392ab22896dbe957f166354cbf9f82856"}
Sep 11 01:27:52.971676 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:52.971266    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7965-371b4e" event={"ID":"aa2bc96b-f4d2-4144-b615-fcfe9b34c859","Type":"ContainerStarted","Data":"398db80914fee0831604b450968a293b1ef18e34274206c059bdfe0252e415cc"}
Sep 11 01:27:52.985864 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:52.985814    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7965-371b4e" podStartSLOduration=1.985796846 podStartE2EDuration="1.985796846s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:27:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:27:52.984891556 +0000 UTC m=+1788.382083730" watchObservedRunningTime="2026-09-11 01:27:52.985796846 +0000 UTC m=+1788.382989020"
Sep 11 01:27:53.571459 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:53.571415    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7965-371b4e"]
Sep 11 01:27:53.574885 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:53.574855    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7965-371b4e"]
Sep 11 01:27:53.975938 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:27:53.975878    2574 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7965-371b4e" container="test-curl-7965-371b4e"
Sep 11 01:27:54.980738 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:54.980692    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7965-371b4e" podUID="aa2bc96b-f4d2-4144-b615-fcfe9b34c859" containerName="test-curl-7965-371b4e" containerID="cri-o://dadc20f3ec30539149db1d1fcaceca0392ab22896dbe957f166354cbf9f82856" gracePeriod=2
Sep 11 01:27:54.982676 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:27:54.982650    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7965-371b4e\" is forbidden: User \"system:node:ip-10-0-139-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-235.ec2.internal' and this object" podUID="aa2bc96b-f4d2-4144-b615-fcfe9b34c859" pod="opendatahub/test-curl-7965-371b4e"
Sep 11 01:27:55.098429 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:27:55.098394    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7965-371b4e\" is forbidden: User \"system:node:ip-10-0-139-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-235.ec2.internal' and this object" podUID="aa2bc96b-f4d2-4144-b615-fcfe9b34c859" pod="opendatahub/test-curl-7965-371b4e"
Sep 11 01:27:55.606923 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:55.606871    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml"]
Sep 11 01:27:55.625973 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:55.625936    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:27:56.709891 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:56.709808    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:27:57.156304 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:57.156279    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7965-371b4e"
Sep 11 01:27:57.234155 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:57.234120    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kn8d7\" (UniqueName: \"kubernetes.io/projected/aa2bc96b-f4d2-4144-b615-fcfe9b34c859-kube-api-access-kn8d7\") pod \"aa2bc96b-f4d2-4144-b615-fcfe9b34c859\" (UID: \"aa2bc96b-f4d2-4144-b615-fcfe9b34c859\") "
Sep 11 01:27:57.236370 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:57.236301    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa2bc96b-f4d2-4144-b615-fcfe9b34c859-kube-api-access-kn8d7" (OuterVolumeSpecName: "kube-api-access-kn8d7") pod "aa2bc96b-f4d2-4144-b615-fcfe9b34c859" (UID: "aa2bc96b-f4d2-4144-b615-fcfe9b34c859"). InnerVolumeSpecName "kube-api-access-kn8d7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:27:57.334895 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:57.334865    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kn8d7\" (UniqueName: \"kubernetes.io/projected/aa2bc96b-f4d2-4144-b615-fcfe9b34c859-kube-api-access-kn8d7\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:27:57.996646 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:57.996614    2574 generic.go:356] "Generic (PLEG): container finished" podID="aa2bc96b-f4d2-4144-b615-fcfe9b34c859" containerID="dadc20f3ec30539149db1d1fcaceca0392ab22896dbe957f166354cbf9f82856" exitCode=137
Sep 11 01:27:57.997150 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:57.996655    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7965-371b4e"
Sep 11 01:27:57.997150 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:57.996715    2574 scope.go:117] "RemoveContainer" containerID="dadc20f3ec30539149db1d1fcaceca0392ab22896dbe957f166354cbf9f82856"
Sep 11 01:27:58.005546 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:58.005530    2574 scope.go:117] "RemoveContainer" containerID="dadc20f3ec30539149db1d1fcaceca0392ab22896dbe957f166354cbf9f82856"
Sep 11 01:27:58.005813 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:27:58.005793    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dadc20f3ec30539149db1d1fcaceca0392ab22896dbe957f166354cbf9f82856\": container with ID starting with dadc20f3ec30539149db1d1fcaceca0392ab22896dbe957f166354cbf9f82856 not found: ID does not exist" containerID="dadc20f3ec30539149db1d1fcaceca0392ab22896dbe957f166354cbf9f82856"
Sep 11 01:27:58.005886 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:58.005820    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dadc20f3ec30539149db1d1fcaceca0392ab22896dbe957f166354cbf9f82856"} err="failed to get container status \"dadc20f3ec30539149db1d1fcaceca0392ab22896dbe957f166354cbf9f82856\": rpc error: code = NotFound desc = could not find container \"dadc20f3ec30539149db1d1fcaceca0392ab22896dbe957f166354cbf9f82856\": container with ID starting with dadc20f3ec30539149db1d1fcaceca0392ab22896dbe957f166354cbf9f82856 not found: ID does not exist"
Sep 11 01:27:58.006702 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:27:58.006680    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7965-371b4e\" is forbidden: User \"system:node:ip-10-0-139-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-235.ec2.internal' and this object" podUID="aa2bc96b-f4d2-4144-b615-fcfe9b34c859" pod="opendatahub/test-curl-7965-371b4e"
Sep 11 01:27:59.099722 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:27:59.099685    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa2bc96b-f4d2-4144-b615-fcfe9b34c859" path="/var/lib/kubelet/pods/aa2bc96b-f4d2-4144-b615-fcfe9b34c859/volumes"
Sep 11 01:28:01.678010 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:01.677966    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6"]
Sep 11 01:28:01.678512 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:01.678437    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa2bc96b-f4d2-4144-b615-fcfe9b34c859" containerName="test-curl-7965-371b4e"
Sep 11 01:28:01.678512 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:01.678453    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa2bc96b-f4d2-4144-b615-fcfe9b34c859" containerName="test-curl-7965-371b4e"
Sep 11 01:28:01.678633 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:01.678528    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa2bc96b-f4d2-4144-b615-fcfe9b34c859" containerName="test-curl-7965-371b4e"
Sep 11 01:28:01.682987 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:01.682964    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6"
Sep 11 01:28:01.685108 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:01.685086    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-d6e26b\"/\"rate-test-model-d6e26b-kserve-self-signed-certs\""
Sep 11 01:28:01.696061 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:01.696021    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6"]
Sep 11 01:28:01.706136 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:01.706108    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:28:01.777127 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:01.777078    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3c1c1f1-9d77-4412-9408-37d6833393b4-tls-certs\") pod \"rate-test-model-d6e26b-kserve-ff4476899-chds6\" (UID: \"f3c1c1f1-9d77-4412-9408-37d6833393b4\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6"
Sep 11 01:28:01.777127 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:01.777131    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3c1c1f1-9d77-4412-9408-37d6833393b4-home\") pod \"rate-test-model-d6e26b-kserve-ff4476899-chds6\" (UID: \"f3c1c1f1-9d77-4412-9408-37d6833393b4\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6"
Sep 11 01:28:01.777371 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:01.777193    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3c1c1f1-9d77-4412-9408-37d6833393b4-tmp-dir\") pod \"rate-test-model-d6e26b-kserve-ff4476899-chds6\" (UID: \"f3c1c1f1-9d77-4412-9408-37d6833393b4\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6"
Sep 11 01:28:01.777371 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:01.777239    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dz5vt\" (UniqueName: \"kubernetes.io/projected/f3c1c1f1-9d77-4412-9408-37d6833393b4-kube-api-access-dz5vt\") pod \"rate-test-model-d6e26b-kserve-ff4476899-chds6\" (UID: \"f3c1c1f1-9d77-4412-9408-37d6833393b4\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6"
Sep 11 01:28:01.777371 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:01.777264    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3c1c1f1-9d77-4412-9408-37d6833393b4-model-cache\") pod \"rate-test-model-d6e26b-kserve-ff4476899-chds6\" (UID: \"f3c1c1f1-9d77-4412-9408-37d6833393b4\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6"
Sep 11 01:28:01.777371 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:01.777286    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3c1c1f1-9d77-4412-9408-37d6833393b4-dshm\") pod \"rate-test-model-d6e26b-kserve-ff4476899-chds6\" (UID: \"f3c1c1f1-9d77-4412-9408-37d6833393b4\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6"
Sep 11 01:28:01.878301 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:01.878256    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3c1c1f1-9d77-4412-9408-37d6833393b4-tls-certs\") pod \"rate-test-model-d6e26b-kserve-ff4476899-chds6\" (UID: \"f3c1c1f1-9d77-4412-9408-37d6833393b4\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6"
Sep 11 01:28:01.878301 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:01.878303    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3c1c1f1-9d77-4412-9408-37d6833393b4-home\") pod \"rate-test-model-d6e26b-kserve-ff4476899-chds6\" (UID: \"f3c1c1f1-9d77-4412-9408-37d6833393b4\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6"
Sep 11 01:28:01.878577 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:01.878337    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3c1c1f1-9d77-4412-9408-37d6833393b4-tmp-dir\") pod \"rate-test-model-d6e26b-kserve-ff4476899-chds6\" (UID: \"f3c1c1f1-9d77-4412-9408-37d6833393b4\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6"
Sep 11 01:28:01.878577 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:01.878372    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dz5vt\" (UniqueName: \"kubernetes.io/projected/f3c1c1f1-9d77-4412-9408-37d6833393b4-kube-api-access-dz5vt\") pod \"rate-test-model-d6e26b-kserve-ff4476899-chds6\" (UID: \"f3c1c1f1-9d77-4412-9408-37d6833393b4\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6"
Sep 11 01:28:01.878577 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:01.878397    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3c1c1f1-9d77-4412-9408-37d6833393b4-model-cache\") pod \"rate-test-model-d6e26b-kserve-ff4476899-chds6\" (UID: \"f3c1c1f1-9d77-4412-9408-37d6833393b4\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6"
Sep 11 01:28:01.878577 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:01.878419    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3c1c1f1-9d77-4412-9408-37d6833393b4-dshm\") pod \"rate-test-model-d6e26b-kserve-ff4476899-chds6\" (UID: \"f3c1c1f1-9d77-4412-9408-37d6833393b4\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6"
Sep 11 01:28:01.878863 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:01.878826    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3c1c1f1-9d77-4412-9408-37d6833393b4-tmp-dir\") pod \"rate-test-model-d6e26b-kserve-ff4476899-chds6\" (UID: \"f3c1c1f1-9d77-4412-9408-37d6833393b4\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6"
Sep 11 01:28:01.878863 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:01.878836    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3c1c1f1-9d77-4412-9408-37d6833393b4-home\") pod \"rate-test-model-d6e26b-kserve-ff4476899-chds6\" (UID: \"f3c1c1f1-9d77-4412-9408-37d6833393b4\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6"
Sep 11 01:28:01.879193 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:01.878885    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3c1c1f1-9d77-4412-9408-37d6833393b4-model-cache\") pod \"rate-test-model-d6e26b-kserve-ff4476899-chds6\" (UID: \"f3c1c1f1-9d77-4412-9408-37d6833393b4\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6"
Sep 11 01:28:01.880860 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:01.880838    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3c1c1f1-9d77-4412-9408-37d6833393b4-dshm\") pod \"rate-test-model-d6e26b-kserve-ff4476899-chds6\" (UID: \"f3c1c1f1-9d77-4412-9408-37d6833393b4\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6"
Sep 11 01:28:01.880998 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:01.880982    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3c1c1f1-9d77-4412-9408-37d6833393b4-tls-certs\") pod \"rate-test-model-d6e26b-kserve-ff4476899-chds6\" (UID: \"f3c1c1f1-9d77-4412-9408-37d6833393b4\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6"
Sep 11 01:28:01.885971 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:01.885900    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dz5vt\" (UniqueName: \"kubernetes.io/projected/f3c1c1f1-9d77-4412-9408-37d6833393b4-kube-api-access-dz5vt\") pod \"rate-test-model-d6e26b-kserve-ff4476899-chds6\" (UID: \"f3c1c1f1-9d77-4412-9408-37d6833393b4\") " pod="ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6"
Sep 11 01:28:01.995367 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:01.995273    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6"
Sep 11 01:28:02.268635 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:02.268611    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6"]
Sep 11 01:28:02.270488 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:28:02.270459    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3c1c1f1_9d77_4412_9408_37d6833393b4.slice/crio-fab3bbab1135ae49c4eb8470358204cec341938186fb1e069fad4c1012eded5e WatchSource:0}: Error finding container fab3bbab1135ae49c4eb8470358204cec341938186fb1e069fad4c1012eded5e: Status 404 returned error can't find the container with id fab3bbab1135ae49c4eb8470358204cec341938186fb1e069fad4c1012eded5e
Sep 11 01:28:03.023818 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:03.023783    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6" event={"ID":"f3c1c1f1-9d77-4412-9408-37d6833393b4","Type":"ContainerStarted","Data":"37c2bafeeab7ce55c08ea7ad6b1ffaa5d8ef3a9d188d7cc5cba8a18a9cdc539d"}
Sep 11 01:28:03.023818 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:03.023826    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6" event={"ID":"f3c1c1f1-9d77-4412-9408-37d6833393b4","Type":"ContainerStarted","Data":"fab3bbab1135ae49c4eb8470358204cec341938186fb1e069fad4c1012eded5e"}
Sep 11 01:28:03.041561 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:03.041337    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6" podStartSLOduration=2.041319647 podStartE2EDuration="2.041319647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:28:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:28:03.040640695 +0000 UTC m=+1798.437832868" watchObservedRunningTime="2026-09-11 01:28:03.041319647 +0000 UTC m=+1798.438511821"
Sep 11 01:28:05.569537 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:05.569503    2574 scope.go:117] "RemoveContainer" containerID="972ea5ada54b92b45b1dd06904e174b003e0a5fa85514437888d9bbf0f086567"
Sep 11 01:28:10.626620 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:10.626572    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml" podUID="0d6b3202-91cc-4444-92c5-72cec717ee2d" containerName="main" containerID="cri-o://d0e2a1da461c3f7b3f9c41d227371de0c3e31fe90c85c750ef7f860a7b083147" gracePeriod=45
Sep 11 01:28:10.880464 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:10.880404    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml"
Sep 11 01:28:10.962136 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:10.962102    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d6b3202-91cc-4444-92c5-72cec717ee2d-tmp-dir\") pod \"0d6b3202-91cc-4444-92c5-72cec717ee2d\" (UID: \"0d6b3202-91cc-4444-92c5-72cec717ee2d\") "
Sep 11 01:28:10.962326 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:10.962146    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d6b3202-91cc-4444-92c5-72cec717ee2d-model-cache\") pod \"0d6b3202-91cc-4444-92c5-72cec717ee2d\" (UID: \"0d6b3202-91cc-4444-92c5-72cec717ee2d\") "
Sep 11 01:28:10.962326 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:10.962168    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zqlbp\" (UniqueName: \"kubernetes.io/projected/0d6b3202-91cc-4444-92c5-72cec717ee2d-kube-api-access-zqlbp\") pod \"0d6b3202-91cc-4444-92c5-72cec717ee2d\" (UID: \"0d6b3202-91cc-4444-92c5-72cec717ee2d\") "
Sep 11 01:28:10.962326 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:10.962201    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d6b3202-91cc-4444-92c5-72cec717ee2d-dshm\") pod \"0d6b3202-91cc-4444-92c5-72cec717ee2d\" (UID: \"0d6b3202-91cc-4444-92c5-72cec717ee2d\") "
Sep 11 01:28:10.962326 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:10.962218    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d6b3202-91cc-4444-92c5-72cec717ee2d-tls-certs\") pod \"0d6b3202-91cc-4444-92c5-72cec717ee2d\" (UID: \"0d6b3202-91cc-4444-92c5-72cec717ee2d\") "
Sep 11 01:28:10.962326 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:10.962251    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d6b3202-91cc-4444-92c5-72cec717ee2d-home\") pod \"0d6b3202-91cc-4444-92c5-72cec717ee2d\" (UID: \"0d6b3202-91cc-4444-92c5-72cec717ee2d\") "
Sep 11 01:28:10.962597 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:10.962453    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d6b3202-91cc-4444-92c5-72cec717ee2d-model-cache" (OuterVolumeSpecName: "model-cache") pod "0d6b3202-91cc-4444-92c5-72cec717ee2d" (UID: "0d6b3202-91cc-4444-92c5-72cec717ee2d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:28:10.962597 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:10.962428    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d6b3202-91cc-4444-92c5-72cec717ee2d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0d6b3202-91cc-4444-92c5-72cec717ee2d" (UID: "0d6b3202-91cc-4444-92c5-72cec717ee2d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:28:10.962597 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:10.962576    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d6b3202-91cc-4444-92c5-72cec717ee2d-tmp-dir\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:28:10.962702 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:10.962598    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d6b3202-91cc-4444-92c5-72cec717ee2d-model-cache\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:28:10.962702 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:10.962625    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d6b3202-91cc-4444-92c5-72cec717ee2d-home" (OuterVolumeSpecName: "home") pod "0d6b3202-91cc-4444-92c5-72cec717ee2d" (UID: "0d6b3202-91cc-4444-92c5-72cec717ee2d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:28:10.964477 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:10.964452    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d6b3202-91cc-4444-92c5-72cec717ee2d-dshm" (OuterVolumeSpecName: "dshm") pod "0d6b3202-91cc-4444-92c5-72cec717ee2d" (UID: "0d6b3202-91cc-4444-92c5-72cec717ee2d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:28:10.964568 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:10.964483    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d6b3202-91cc-4444-92c5-72cec717ee2d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0d6b3202-91cc-4444-92c5-72cec717ee2d" (UID: "0d6b3202-91cc-4444-92c5-72cec717ee2d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:28:10.964568 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:10.964543    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d6b3202-91cc-4444-92c5-72cec717ee2d-kube-api-access-zqlbp" (OuterVolumeSpecName: "kube-api-access-zqlbp") pod "0d6b3202-91cc-4444-92c5-72cec717ee2d" (UID: "0d6b3202-91cc-4444-92c5-72cec717ee2d"). InnerVolumeSpecName "kube-api-access-zqlbp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:28:11.063165 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:11.063134    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d6b3202-91cc-4444-92c5-72cec717ee2d-home\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:28:11.063165 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:11.063157    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqlbp\" (UniqueName: \"kubernetes.io/projected/0d6b3202-91cc-4444-92c5-72cec717ee2d-kube-api-access-zqlbp\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:28:11.063165 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:11.063168    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d6b3202-91cc-4444-92c5-72cec717ee2d-dshm\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:28:11.063390 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:11.063176    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d6b3202-91cc-4444-92c5-72cec717ee2d-tls-certs\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:28:11.064633 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:11.064609    2574 generic.go:356] "Generic (PLEG): container finished" podID="0d6b3202-91cc-4444-92c5-72cec717ee2d" containerID="d0e2a1da461c3f7b3f9c41d227371de0c3e31fe90c85c750ef7f860a7b083147" exitCode=0
Sep 11 01:28:11.064752 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:11.064653    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml"
Sep 11 01:28:11.064752 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:11.064683    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml" event={"ID":"0d6b3202-91cc-4444-92c5-72cec717ee2d","Type":"ContainerDied","Data":"d0e2a1da461c3f7b3f9c41d227371de0c3e31fe90c85c750ef7f860a7b083147"}
Sep 11 01:28:11.064752 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:11.064727    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml" event={"ID":"0d6b3202-91cc-4444-92c5-72cec717ee2d","Type":"ContainerDied","Data":"5e72711974c9db08c0311eec36bb39cace8de1486fb728a1833634eb3d0f324f"}
Sep 11 01:28:11.064752 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:11.064748    2574 scope.go:117] "RemoveContainer" containerID="d0e2a1da461c3f7b3f9c41d227371de0c3e31fe90c85c750ef7f860a7b083147"
Sep 11 01:28:11.074314 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:11.074298    2574 scope.go:117] "RemoveContainer" containerID="d0e2a1da461c3f7b3f9c41d227371de0c3e31fe90c85c750ef7f860a7b083147"
Sep 11 01:28:11.074564 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:28:11.074548    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0e2a1da461c3f7b3f9c41d227371de0c3e31fe90c85c750ef7f860a7b083147\": container with ID starting with d0e2a1da461c3f7b3f9c41d227371de0c3e31fe90c85c750ef7f860a7b083147 not found: ID does not exist" containerID="d0e2a1da461c3f7b3f9c41d227371de0c3e31fe90c85c750ef7f860a7b083147"
Sep 11 01:28:11.074610 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:11.074572    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0e2a1da461c3f7b3f9c41d227371de0c3e31fe90c85c750ef7f860a7b083147"} err="failed to get container status \"d0e2a1da461c3f7b3f9c41d227371de0c3e31fe90c85c750ef7f860a7b083147\": rpc error: code = NotFound desc = could not find container \"d0e2a1da461c3f7b3f9c41d227371de0c3e31fe90c85c750ef7f860a7b083147\": container with ID starting with d0e2a1da461c3f7b3f9c41d227371de0c3e31fe90c85c750ef7f860a7b083147 not found: ID does not exist"
Sep 11 01:28:11.084631 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:11.084609    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml"]
Sep 11 01:28:11.090606 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:11.090584    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-d6e26b/sub-test-model-d6e26b-kserve-c6cccc7fc-8l8ml"]
Sep 11 01:28:11.100083 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:11.100061    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d6b3202-91cc-4444-92c5-72cec717ee2d" path="/var/lib/kubelet/pods/0d6b3202-91cc-4444-92c5-72cec717ee2d/volumes"
Sep 11 01:28:11.995610 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:11.995570    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6"
Sep 11 01:28:11.996141 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:11.995834    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6"
Sep 11 01:28:12.007891 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:12.007872    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6"
Sep 11 01:28:12.082586 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:12.082560    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6"
Sep 11 01:28:27.012261 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:27.012214    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:28:50.031228 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:28:50.031192    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:29:05.596062 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:05.596031    2574 scope.go:117] "RemoveContainer" containerID="afc3595def1c8f42b89012181c1945e10730fe9a525137c33e8ea9ea93449e63"
Sep 11 01:29:11.216965 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:11.216934    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6"]
Sep 11 01:29:11.229530 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:11.229502    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:29:12.184613 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:12.184581    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:29:12.635059 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:12.634984    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:29:12.944276 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:12.944245    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:29:13.251561 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:13.251476    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:29:15.562692 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:15.562652    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:29:15.654091 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:15.654061    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:29:16.722537 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:16.722500    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:29:16.826290 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:16.826260    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:29:17.071542 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:17.071464    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:29:17.304717 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:17.304685    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:29:17.416565 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:17.416522    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:29:17.472705 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:17.472674    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c1edc6-86d99bdbf8-8hhms"]
Sep 11 01:29:17.473050 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:17.473037    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d6b3202-91cc-4444-92c5-72cec717ee2d" containerName="main"
Sep 11 01:29:17.473101 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:17.473052    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d6b3202-91cc-4444-92c5-72cec717ee2d" containerName="main"
Sep 11 01:29:17.473138 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:17.473114    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d6b3202-91cc-4444-92c5-72cec717ee2d" containerName="main"
Sep 11 01:29:17.475852 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:17.475836    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c1edc6-86d99bdbf8-8hhms"
Sep 11 01:29:17.478135 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:17.478112    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-c1edc6-dockercfg-bqk6d\""
Sep 11 01:29:17.478245 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:17.478114    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-c1edc6\""
Sep 11 01:29:17.490146 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:17.490121    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c1edc6-86d99bdbf8-8hhms"]
Sep 11 01:29:17.538607 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:17.538580    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrz6j\" (UniqueName: \"kubernetes.io/projected/7e8ddd83-87bd-443c-b068-e3bff6af6654-kube-api-access-lrz6j\") pod \"payload-pre-processing-e2e-ipp-a-c1edc6-86d99bdbf8-8hhms\" (UID: \"7e8ddd83-87bd-443c-b068-e3bff6af6654\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c1edc6-86d99bdbf8-8hhms"
Sep 11 01:29:17.538786 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:17.538619    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e8ddd83-87bd-443c-b068-e3bff6af6654-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-c1edc6-86d99bdbf8-8hhms\" (UID: \"7e8ddd83-87bd-443c-b068-e3bff6af6654\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c1edc6-86d99bdbf8-8hhms"
Sep 11 01:29:17.559089 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:17.559046    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c1edc6-7c6f5584df-f5pkt"]
Sep 11 01:29:17.562134 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:17.562120    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-c1edc6-7c6f5584df-f5pkt"
Sep 11 01:29:17.572732 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:17.572710    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c1edc6-7c6f5584df-f5pkt"]
Sep 11 01:29:17.640047 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:17.640017    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e8ddd83-87bd-443c-b068-e3bff6af6654-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-c1edc6-86d99bdbf8-8hhms\" (UID: \"7e8ddd83-87bd-443c-b068-e3bff6af6654\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c1edc6-86d99bdbf8-8hhms"
Sep 11 01:29:17.640206 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:17.640061    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2dwb\" (UniqueName: \"kubernetes.io/projected/bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b-kube-api-access-r2dwb\") pod \"payload-processing-e2e-ipp-a-c1edc6-7c6f5584df-f5pkt\" (UID: \"bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c1edc6-7c6f5584df-f5pkt"
Sep 11 01:29:17.640206 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:17.640138    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrz6j\" (UniqueName: \"kubernetes.io/projected/7e8ddd83-87bd-443c-b068-e3bff6af6654-kube-api-access-lrz6j\") pod \"payload-pre-processing-e2e-ipp-a-c1edc6-86d99bdbf8-8hhms\" (UID: \"7e8ddd83-87bd-443c-b068-e3bff6af6654\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c1edc6-86d99bdbf8-8hhms"
Sep 11 01:29:17.640206 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:17.640160    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-c1edc6-7c6f5584df-f5pkt\" (UID: \"bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c1edc6-7c6f5584df-f5pkt"
Sep 11 01:29:17.640691 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:17.640671    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e8ddd83-87bd-443c-b068-e3bff6af6654-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-c1edc6-86d99bdbf8-8hhms\" (UID: \"7e8ddd83-87bd-443c-b068-e3bff6af6654\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c1edc6-86d99bdbf8-8hhms"
Sep 11 01:29:17.647670 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:17.647641    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrz6j\" (UniqueName: \"kubernetes.io/projected/7e8ddd83-87bd-443c-b068-e3bff6af6654-kube-api-access-lrz6j\") pod \"payload-pre-processing-e2e-ipp-a-c1edc6-86d99bdbf8-8hhms\" (UID: \"7e8ddd83-87bd-443c-b068-e3bff6af6654\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c1edc6-86d99bdbf8-8hhms"
Sep 11 01:29:17.740725 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:17.740644    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-c1edc6-7c6f5584df-f5pkt\" (UID: \"bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c1edc6-7c6f5584df-f5pkt"
Sep 11 01:29:17.740725 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:17.740695    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2dwb\" (UniqueName: \"kubernetes.io/projected/bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b-kube-api-access-r2dwb\") pod \"payload-processing-e2e-ipp-a-c1edc6-7c6f5584df-f5pkt\" (UID: \"bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c1edc6-7c6f5584df-f5pkt"
Sep 11 01:29:17.741470 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:17.741446    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-c1edc6-7c6f5584df-f5pkt\" (UID: \"bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c1edc6-7c6f5584df-f5pkt"
Sep 11 01:29:17.748325 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:17.748294    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2dwb\" (UniqueName: \"kubernetes.io/projected/bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b-kube-api-access-r2dwb\") pod \"payload-processing-e2e-ipp-a-c1edc6-7c6f5584df-f5pkt\" (UID: \"bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c1edc6-7c6f5584df-f5pkt"
Sep 11 01:29:17.786160 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:17.786122    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c1edc6-86d99bdbf8-8hhms"
Sep 11 01:29:17.872469 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:17.872440    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-c1edc6-7c6f5584df-f5pkt"
Sep 11 01:29:18.056571 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:18.056546    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c1edc6-86d99bdbf8-8hhms"]
Sep 11 01:29:18.059250 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:29:18.059218    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e8ddd83_87bd_443c_b068_e3bff6af6654.slice/crio-ce16c9743ae3b9d0ba5266c0f8ce5ab5070073bbd2b280c2e6f920faa6e05a82 WatchSource:0}: Error finding container ce16c9743ae3b9d0ba5266c0f8ce5ab5070073bbd2b280c2e6f920faa6e05a82: Status 404 returned error can't find the container with id ce16c9743ae3b9d0ba5266c0f8ce5ab5070073bbd2b280c2e6f920faa6e05a82
Sep 11 01:29:18.145582 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:18.145556    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c1edc6-7c6f5584df-f5pkt"]
Sep 11 01:29:18.147588 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:29:18.147553    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbed1b6f2_b6ad_42d4_bd46_1e3e7509cf1b.slice/crio-09367f14dd0d9af08fe57fc16cbe860ea0073f6079bdbfb24525accb80970fa5 WatchSource:0}: Error finding container 09367f14dd0d9af08fe57fc16cbe860ea0073f6079bdbfb24525accb80970fa5: Status 404 returned error can't find the container with id 09367f14dd0d9af08fe57fc16cbe860ea0073f6079bdbfb24525accb80970fa5
Sep 11 01:29:18.388971 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:18.388770    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c1edc6-86d99bdbf8-8hhms" event={"ID":"7e8ddd83-87bd-443c-b068-e3bff6af6654","Type":"ContainerStarted","Data":"1d3961b92797337a24f5ed11d3bfa7b68c1865c68159fec84d6e654a25462821"}
Sep 11 01:29:18.388971 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:18.388838    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c1edc6-86d99bdbf8-8hhms"
Sep 11 01:29:18.388971 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:18.388861    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c1edc6-86d99bdbf8-8hhms" event={"ID":"7e8ddd83-87bd-443c-b068-e3bff6af6654","Type":"ContainerStarted","Data":"ce16c9743ae3b9d0ba5266c0f8ce5ab5070073bbd2b280c2e6f920faa6e05a82"}
Sep 11 01:29:18.390658 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:18.390626    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-c1edc6-7c6f5584df-f5pkt" event={"ID":"bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b","Type":"ContainerStarted","Data":"869480a80310c9925abb59ad03497bc9ade6fbb3383ab49a249b87f5035b7790"}
Sep 11 01:29:18.390802 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:18.390682    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-c1edc6-7c6f5584df-f5pkt" event={"ID":"bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b","Type":"ContainerStarted","Data":"09367f14dd0d9af08fe57fc16cbe860ea0073f6079bdbfb24525accb80970fa5"}
Sep 11 01:29:18.390802 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:18.390712    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-c1edc6-7c6f5584df-f5pkt"
Sep 11 01:29:18.405396 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:18.405347    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c1edc6-86d99bdbf8-8hhms" podStartSLOduration=1.4053296 podStartE2EDuration="1.4053296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:29:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:29:18.402658173 +0000 UTC m=+1873.799850346" watchObservedRunningTime="2026-09-11 01:29:18.4053296 +0000 UTC m=+1873.802521777"
Sep 11 01:29:18.419612 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:18.419562    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-c1edc6-7c6f5584df-f5pkt" podStartSLOduration=1.419546692 podStartE2EDuration="1.419546692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:29:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:29:18.416881871 +0000 UTC m=+1873.814074048" watchObservedRunningTime="2026-09-11 01:29:18.419546692 +0000 UTC m=+1873.816738868"
Sep 11 01:29:26.236194 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:26.236128    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6" podUID="f3c1c1f1-9d77-4412-9408-37d6833393b4" containerName="main" containerID="cri-o://37c2bafeeab7ce55c08ea7ad6b1ffaa5d8ef3a9d188d7cc5cba8a18a9cdc539d" gracePeriod=45
Sep 11 01:29:26.435021 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:26.434978    2574 generic.go:356] "Generic (PLEG): container finished" podID="f3c1c1f1-9d77-4412-9408-37d6833393b4" containerID="37c2bafeeab7ce55c08ea7ad6b1ffaa5d8ef3a9d188d7cc5cba8a18a9cdc539d" exitCode=0
Sep 11 01:29:26.435154 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:26.435059    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6" event={"ID":"f3c1c1f1-9d77-4412-9408-37d6833393b4","Type":"ContainerDied","Data":"37c2bafeeab7ce55c08ea7ad6b1ffaa5d8ef3a9d188d7cc5cba8a18a9cdc539d"}
Sep 11 01:29:26.478324 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:26.478297    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6"
Sep 11 01:29:26.614239 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:26.614151    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3c1c1f1-9d77-4412-9408-37d6833393b4-home\") pod \"f3c1c1f1-9d77-4412-9408-37d6833393b4\" (UID: \"f3c1c1f1-9d77-4412-9408-37d6833393b4\") "
Sep 11 01:29:26.614239 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:26.614194    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3c1c1f1-9d77-4412-9408-37d6833393b4-tls-certs\") pod \"f3c1c1f1-9d77-4412-9408-37d6833393b4\" (UID: \"f3c1c1f1-9d77-4412-9408-37d6833393b4\") "
Sep 11 01:29:26.614483 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:26.614257    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3c1c1f1-9d77-4412-9408-37d6833393b4-tmp-dir\") pod \"f3c1c1f1-9d77-4412-9408-37d6833393b4\" (UID: \"f3c1c1f1-9d77-4412-9408-37d6833393b4\") "
Sep 11 01:29:26.614483 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:26.614304    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3c1c1f1-9d77-4412-9408-37d6833393b4-dshm\") pod \"f3c1c1f1-9d77-4412-9408-37d6833393b4\" (UID: \"f3c1c1f1-9d77-4412-9408-37d6833393b4\") "
Sep 11 01:29:26.614483 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:26.614344    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dz5vt\" (UniqueName: \"kubernetes.io/projected/f3c1c1f1-9d77-4412-9408-37d6833393b4-kube-api-access-dz5vt\") pod \"f3c1c1f1-9d77-4412-9408-37d6833393b4\" (UID: \"f3c1c1f1-9d77-4412-9408-37d6833393b4\") "
Sep 11 01:29:26.614483 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:26.614383    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3c1c1f1-9d77-4412-9408-37d6833393b4-model-cache\") pod \"f3c1c1f1-9d77-4412-9408-37d6833393b4\" (UID: \"f3c1c1f1-9d77-4412-9408-37d6833393b4\") "
Sep 11 01:29:26.614680 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:26.614477    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3c1c1f1-9d77-4412-9408-37d6833393b4-home" (OuterVolumeSpecName: "home") pod "f3c1c1f1-9d77-4412-9408-37d6833393b4" (UID: "f3c1c1f1-9d77-4412-9408-37d6833393b4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:29:26.614680 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:26.614570    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3c1c1f1-9d77-4412-9408-37d6833393b4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f3c1c1f1-9d77-4412-9408-37d6833393b4" (UID: "f3c1c1f1-9d77-4412-9408-37d6833393b4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:29:26.614751 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:26.614719    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3c1c1f1-9d77-4412-9408-37d6833393b4-model-cache" (OuterVolumeSpecName: "model-cache") pod "f3c1c1f1-9d77-4412-9408-37d6833393b4" (UID: "f3c1c1f1-9d77-4412-9408-37d6833393b4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:29:26.614751 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:26.614736    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3c1c1f1-9d77-4412-9408-37d6833393b4-home\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:29:26.614819 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:26.614751    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3c1c1f1-9d77-4412-9408-37d6833393b4-tmp-dir\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:29:26.616549 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:26.616522    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3c1c1f1-9d77-4412-9408-37d6833393b4-dshm" (OuterVolumeSpecName: "dshm") pod "f3c1c1f1-9d77-4412-9408-37d6833393b4" (UID: "f3c1c1f1-9d77-4412-9408-37d6833393b4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:29:26.616678 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:26.616590    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3c1c1f1-9d77-4412-9408-37d6833393b4-kube-api-access-dz5vt" (OuterVolumeSpecName: "kube-api-access-dz5vt") pod "f3c1c1f1-9d77-4412-9408-37d6833393b4" (UID: "f3c1c1f1-9d77-4412-9408-37d6833393b4"). InnerVolumeSpecName "kube-api-access-dz5vt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:29:26.616741 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:26.616693    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3c1c1f1-9d77-4412-9408-37d6833393b4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f3c1c1f1-9d77-4412-9408-37d6833393b4" (UID: "f3c1c1f1-9d77-4412-9408-37d6833393b4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:29:26.716048 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:26.716019    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3c1c1f1-9d77-4412-9408-37d6833393b4-tls-certs\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:29:26.716048 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:26.716043    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3c1c1f1-9d77-4412-9408-37d6833393b4-dshm\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:29:26.716048 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:26.716052    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dz5vt\" (UniqueName: \"kubernetes.io/projected/f3c1c1f1-9d77-4412-9408-37d6833393b4-kube-api-access-dz5vt\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:29:26.716250 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:26.716060    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3c1c1f1-9d77-4412-9408-37d6833393b4-model-cache\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:29:27.440953 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:27.440904    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6"
Sep 11 01:29:27.441363 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:27.440951    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6" event={"ID":"f3c1c1f1-9d77-4412-9408-37d6833393b4","Type":"ContainerDied","Data":"fab3bbab1135ae49c4eb8470358204cec341938186fb1e069fad4c1012eded5e"}
Sep 11 01:29:27.441363 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:27.441008    2574 scope.go:117] "RemoveContainer" containerID="37c2bafeeab7ce55c08ea7ad6b1ffaa5d8ef3a9d188d7cc5cba8a18a9cdc539d"
Sep 11 01:29:27.456768 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:27.456742    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6"]
Sep 11 01:29:27.458407 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:27.458378    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-d6e26b/rate-test-model-d6e26b-kserve-ff4476899-chds6"]
Sep 11 01:29:29.099774 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:29.099743    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3c1c1f1-9d77-4412-9408-37d6833393b4" path="/var/lib/kubelet/pods/f3c1c1f1-9d77-4412-9408-37d6833393b4/volumes"
Sep 11 01:29:29.397646 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:29.397576    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-c1edc6-7c6f5584df-f5pkt"
Sep 11 01:29:29.397646 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:29.397638    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c1edc6-86d99bdbf8-8hhms"
Sep 11 01:29:39.071932 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:39.071874    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:29:40.598512 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:40.598477    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:29:41.348410 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:41.348378    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:29:41.636510 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:41.636434    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:29:41.839630 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:41.839599    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:29:44.775707 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:44.775670    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:29:44.875860 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:44.875827    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:29:45.933120 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:45.933079    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:29:46.037470 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:46.037438    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:29:46.135174 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:46.135146    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:29:46.406671 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:46.406592    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:29:46.512244 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:46.512204    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:29:46.520222 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:46.520196    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-40f90a-58cb6f886b-xmwmb"]
Sep 11 01:29:46.520575 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:46.520564    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3c1c1f1-9d77-4412-9408-37d6833393b4" containerName="main"
Sep 11 01:29:46.520663 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:46.520577    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3c1c1f1-9d77-4412-9408-37d6833393b4" containerName="main"
Sep 11 01:29:46.520663 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:46.520640    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3c1c1f1-9d77-4412-9408-37d6833393b4" containerName="main"
Sep 11 01:29:46.523590 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:46.523575    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-40f90a-58cb6f886b-xmwmb"
Sep 11 01:29:46.526705 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:46.526683    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-40f90a-dockercfg-mf62b\""
Sep 11 01:29:46.527363 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:46.527341    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-40f90a\""
Sep 11 01:29:46.540495 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:46.540471    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-40f90a-58cb6f886b-xmwmb"]
Sep 11 01:29:46.542562 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:46.542540    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-40f90a-6b7f8c5df9-fpjwc"]
Sep 11 01:29:46.548644 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:46.548627    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-40f90a-6b7f8c5df9-fpjwc"
Sep 11 01:29:46.557335 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:46.557304    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-40f90a-6b7f8c5df9-fpjwc"]
Sep 11 01:29:46.685177 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:46.685152    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlkh9\" (UniqueName: \"kubernetes.io/projected/9932705f-5b73-4290-b6d0-6fbef66a72d0-kube-api-access-nlkh9\") pod \"payload-pre-processing-e2e-ipp-b-40f90a-58cb6f886b-xmwmb\" (UID: \"9932705f-5b73-4290-b6d0-6fbef66a72d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-40f90a-58cb6f886b-xmwmb"
Sep 11 01:29:46.685310 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:46.685220    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9932705f-5b73-4290-b6d0-6fbef66a72d0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-40f90a-58cb6f886b-xmwmb\" (UID: \"9932705f-5b73-4290-b6d0-6fbef66a72d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-40f90a-58cb6f886b-xmwmb"
Sep 11 01:29:46.685310 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:46.685286    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99909fa4-b502-45ba-a93e-433e11679741-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-40f90a-6b7f8c5df9-fpjwc\" (UID: \"99909fa4-b502-45ba-a93e-433e11679741\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-40f90a-6b7f8c5df9-fpjwc"
Sep 11 01:29:46.685310 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:46.685308    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zwdq\" (UniqueName: \"kubernetes.io/projected/99909fa4-b502-45ba-a93e-433e11679741-kube-api-access-7zwdq\") pod \"payload-processing-e2e-ipp-b-40f90a-6b7f8c5df9-fpjwc\" (UID: \"99909fa4-b502-45ba-a93e-433e11679741\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-40f90a-6b7f8c5df9-fpjwc"
Sep 11 01:29:46.785923 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:46.785884    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9932705f-5b73-4290-b6d0-6fbef66a72d0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-40f90a-58cb6f886b-xmwmb\" (UID: \"9932705f-5b73-4290-b6d0-6fbef66a72d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-40f90a-58cb6f886b-xmwmb"
Sep 11 01:29:46.786099 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:46.785995    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99909fa4-b502-45ba-a93e-433e11679741-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-40f90a-6b7f8c5df9-fpjwc\" (UID: \"99909fa4-b502-45ba-a93e-433e11679741\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-40f90a-6b7f8c5df9-fpjwc"
Sep 11 01:29:46.786099 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:46.786031    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zwdq\" (UniqueName: \"kubernetes.io/projected/99909fa4-b502-45ba-a93e-433e11679741-kube-api-access-7zwdq\") pod \"payload-processing-e2e-ipp-b-40f90a-6b7f8c5df9-fpjwc\" (UID: \"99909fa4-b502-45ba-a93e-433e11679741\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-40f90a-6b7f8c5df9-fpjwc"
Sep 11 01:29:46.786099 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:46.786070    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlkh9\" (UniqueName: \"kubernetes.io/projected/9932705f-5b73-4290-b6d0-6fbef66a72d0-kube-api-access-nlkh9\") pod \"payload-pre-processing-e2e-ipp-b-40f90a-58cb6f886b-xmwmb\" (UID: \"9932705f-5b73-4290-b6d0-6fbef66a72d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-40f90a-58cb6f886b-xmwmb"
Sep 11 01:29:46.786825 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:46.786802    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9932705f-5b73-4290-b6d0-6fbef66a72d0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-40f90a-58cb6f886b-xmwmb\" (UID: \"9932705f-5b73-4290-b6d0-6fbef66a72d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-40f90a-58cb6f886b-xmwmb"
Sep 11 01:29:46.786884 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:46.786802    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99909fa4-b502-45ba-a93e-433e11679741-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-40f90a-6b7f8c5df9-fpjwc\" (UID: \"99909fa4-b502-45ba-a93e-433e11679741\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-40f90a-6b7f8c5df9-fpjwc"
Sep 11 01:29:46.795612 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:46.795589    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlkh9\" (UniqueName: \"kubernetes.io/projected/9932705f-5b73-4290-b6d0-6fbef66a72d0-kube-api-access-nlkh9\") pod \"payload-pre-processing-e2e-ipp-b-40f90a-58cb6f886b-xmwmb\" (UID: \"9932705f-5b73-4290-b6d0-6fbef66a72d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-40f90a-58cb6f886b-xmwmb"
Sep 11 01:29:46.795735 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:46.795647    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zwdq\" (UniqueName: \"kubernetes.io/projected/99909fa4-b502-45ba-a93e-433e11679741-kube-api-access-7zwdq\") pod \"payload-processing-e2e-ipp-b-40f90a-6b7f8c5df9-fpjwc\" (UID: \"99909fa4-b502-45ba-a93e-433e11679741\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-40f90a-6b7f8c5df9-fpjwc"
Sep 11 01:29:46.833686 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:46.833655    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-40f90a-58cb6f886b-xmwmb"
Sep 11 01:29:46.859729 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:46.859693    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-40f90a-6b7f8c5df9-fpjwc"
Sep 11 01:29:47.111011 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:47.110983    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-40f90a-58cb6f886b-xmwmb"]
Sep 11 01:29:47.113072 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:29:47.113040    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9932705f_5b73_4290_b6d0_6fbef66a72d0.slice/crio-4d83191cb90a577c238daf790fba3a3be74122453acc606cf46ac5d15e34dec1 WatchSource:0}: Error finding container 4d83191cb90a577c238daf790fba3a3be74122453acc606cf46ac5d15e34dec1: Status 404 returned error can't find the container with id 4d83191cb90a577c238daf790fba3a3be74122453acc606cf46ac5d15e34dec1
Sep 11 01:29:47.146701 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:47.146675    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-40f90a-6b7f8c5df9-fpjwc"]
Sep 11 01:29:47.149044 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:29:47.149016    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99909fa4_b502_45ba_a93e_433e11679741.slice/crio-16afc56b6f36a6ce2a315da2faee221e5ce6ce3ef58dcf26123cae2e41e751b7 WatchSource:0}: Error finding container 16afc56b6f36a6ce2a315da2faee221e5ce6ce3ef58dcf26123cae2e41e751b7: Status 404 returned error can't find the container with id 16afc56b6f36a6ce2a315da2faee221e5ce6ce3ef58dcf26123cae2e41e751b7
Sep 11 01:29:47.537814 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:47.537773    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-40f90a-6b7f8c5df9-fpjwc" event={"ID":"99909fa4-b502-45ba-a93e-433e11679741","Type":"ContainerStarted","Data":"2ac08ce3e6a9e9b949be6a10f8b646af3f62c4cf286c33610a5eb8be96fc904f"}
Sep 11 01:29:47.538055 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:47.537861    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-40f90a-6b7f8c5df9-fpjwc"
Sep 11 01:29:47.538055 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:47.537894    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-40f90a-6b7f8c5df9-fpjwc" event={"ID":"99909fa4-b502-45ba-a93e-433e11679741","Type":"ContainerStarted","Data":"16afc56b6f36a6ce2a315da2faee221e5ce6ce3ef58dcf26123cae2e41e751b7"}
Sep 11 01:29:47.539501 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:47.539475    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-40f90a-58cb6f886b-xmwmb" event={"ID":"9932705f-5b73-4290-b6d0-6fbef66a72d0","Type":"ContainerStarted","Data":"599d1005a5678ed3c83e679c81542fd613afd409daa609953f6e50de12ab7524"}
Sep 11 01:29:47.539611 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:47.539522    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-40f90a-58cb6f886b-xmwmb" event={"ID":"9932705f-5b73-4290-b6d0-6fbef66a72d0","Type":"ContainerStarted","Data":"4d83191cb90a577c238daf790fba3a3be74122453acc606cf46ac5d15e34dec1"}
Sep 11 01:29:47.539611 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:47.539561    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-40f90a-58cb6f886b-xmwmb"
Sep 11 01:29:47.555983 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:47.555902    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-40f90a-6b7f8c5df9-fpjwc" podStartSLOduration=1.555885757 podStartE2EDuration="1.555885757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:29:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:29:47.554045675 +0000 UTC m=+1902.951237850" watchObservedRunningTime="2026-09-11 01:29:47.555885757 +0000 UTC m=+1902.953077931"
Sep 11 01:29:47.571871 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:47.571827    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-40f90a-58cb6f886b-xmwmb" podStartSLOduration=1.5718125029999999 podStartE2EDuration="1.571812503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:29:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:29:47.56994906 +0000 UTC m=+1902.967141237" watchObservedRunningTime="2026-09-11 01:29:47.571812503 +0000 UTC m=+1902.969004676"
Sep 11 01:29:51.335640 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:51.335607    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:29:51.396846 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:51.396819    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:29:58.546777 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:58.546743    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-40f90a-6b7f8c5df9-fpjwc"
Sep 11 01:29:58.547204 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:29:58.546965    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-40f90a-58cb6f886b-xmwmb"
Sep 11 01:30:00.149193 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:00.149164    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29818170-wl98m"]
Sep 11 01:30:00.153974 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:00.153947    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b9930f-29818170-5gflq"]
Sep 11 01:30:00.154121 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:00.154059    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818170-wl98m"
Sep 11 01:30:00.157219 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:00.157195    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q6gbn\""
Sep 11 01:30:00.157614 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:00.157597    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b9930f-29818170-5gflq"
Sep 11 01:30:00.157985 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:00.157962    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-40f90a-29818170-bzz4w"]
Sep 11 01:30:00.161756 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:00.161740    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-40f90a-29818170-bzz4w"
Sep 11 01:30:00.167352 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:00.167334    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818170-wl98m"
Sep 11 01:30:00.177397 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:00.177330    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29818170-wl98m"]
Sep 11 01:30:00.178798 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:00.178775    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b9930f-29818170-5gflq"]
Sep 11 01:30:00.179644 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:00.179593    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b9930f-29818170-5gflq"
Sep 11 01:30:00.199944 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:00.199920    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-40f90a-29818170-bzz4w"]
Sep 11 01:30:00.214811 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:00.214790    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-40f90a-29818170-bzz4w"
Sep 11 01:30:00.457932 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:00.457884    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29818170-wl98m"]
Sep 11 01:30:00.460281 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:30:00.460243    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbab7a05_c5ce_4039_a867_24421fd51d48.slice/crio-38853ab8e5da34fd40d47315d4def91f4a620f3491bb509f1db15d49a138211d WatchSource:0}: Error finding container 38853ab8e5da34fd40d47315d4def91f4a620f3491bb509f1db15d49a138211d: Status 404 returned error can't find the container with id 38853ab8e5da34fd40d47315d4def91f4a620f3491bb509f1db15d49a138211d
Sep 11 01:30:00.462761 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:00.462735    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b9930f-29818170-5gflq"]
Sep 11 01:30:00.464996 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:30:00.464971    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8795cde5_6a3a_4201_bb24_c0d3bfe1c1a8.slice/crio-680e9ae240652c9226c3bc28aa498f485244aaa8c2ba13d0200fef42133bd30c WatchSource:0}: Error finding container 680e9ae240652c9226c3bc28aa498f485244aaa8c2ba13d0200fef42133bd30c: Status 404 returned error can't find the container with id 680e9ae240652c9226c3bc28aa498f485244aaa8c2ba13d0200fef42133bd30c
Sep 11 01:30:00.504881 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:00.504856    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-40f90a-29818170-bzz4w"]
Sep 11 01:30:00.506726 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:30:00.506703    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab95ce39_5e6c_40b4_8b64_cfe7c01fb401.slice/crio-28b3a981d730363301b2891075853e099b4c9b74955b2731d15c16fa64dd0f54 WatchSource:0}: Error finding container 28b3a981d730363301b2891075853e099b4c9b74955b2731d15c16fa64dd0f54: Status 404 returned error can't find the container with id 28b3a981d730363301b2891075853e099b4c9b74955b2731d15c16fa64dd0f54
Sep 11 01:30:00.604839 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:00.604805    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b9930f-29818170-5gflq" event={"ID":"8795cde5-6a3a-4201-bb24-c0d3bfe1c1a8","Type":"ContainerStarted","Data":"680e9ae240652c9226c3bc28aa498f485244aaa8c2ba13d0200fef42133bd30c"}
Sep 11 01:30:00.605971 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:00.605949    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818170-wl98m" event={"ID":"bbab7a05-c5ce-4039-a867-24421fd51d48","Type":"ContainerStarted","Data":"38853ab8e5da34fd40d47315d4def91f4a620f3491bb509f1db15d49a138211d"}
Sep 11 01:30:00.607097 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:00.607079    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-40f90a-29818170-bzz4w" event={"ID":"ab95ce39-5e6c-40b4-8b64-cfe7c01fb401","Type":"ContainerStarted","Data":"28b3a981d730363301b2891075853e099b4c9b74955b2731d15c16fa64dd0f54"}
Sep 11 01:30:01.614963 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:01.614900    2574 generic.go:356] "Generic (PLEG): container finished" podID="ab95ce39-5e6c-40b4-8b64-cfe7c01fb401" containerID="1203bb437eadcbe7a38baea86bd0ef45318c1625b4db1d9cebc97f9f5a6f1990" exitCode=0
Sep 11 01:30:01.615356 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:01.614993    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-40f90a-29818170-bzz4w" event={"ID":"ab95ce39-5e6c-40b4-8b64-cfe7c01fb401","Type":"ContainerDied","Data":"1203bb437eadcbe7a38baea86bd0ef45318c1625b4db1d9cebc97f9f5a6f1990"}
Sep 11 01:30:01.616578 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:01.616553    2574 generic.go:356] "Generic (PLEG): container finished" podID="8795cde5-6a3a-4201-bb24-c0d3bfe1c1a8" containerID="5e518a1391d20ec5bc8b4b1d3042720661d51f1100a82332d7cf916f6a61f63a" exitCode=0
Sep 11 01:30:01.616702 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:01.616635    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b9930f-29818170-5gflq" event={"ID":"8795cde5-6a3a-4201-bb24-c0d3bfe1c1a8","Type":"ContainerDied","Data":"5e518a1391d20ec5bc8b4b1d3042720661d51f1100a82332d7cf916f6a61f63a"}
Sep 11 01:30:01.618188 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:01.618160    2574 generic.go:356] "Generic (PLEG): container finished" podID="bbab7a05-c5ce-4039-a867-24421fd51d48" containerID="0b5ec5ca2d485f665a033b3871fb416050cc8511f4a06d1ed298148fa6f7db2f" exitCode=0
Sep 11 01:30:01.618295 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:01.618200    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818170-wl98m" event={"ID":"bbab7a05-c5ce-4039-a867-24421fd51d48","Type":"ContainerDied","Data":"0b5ec5ca2d485f665a033b3871fb416050cc8511f4a06d1ed298148fa6f7db2f"}
Sep 11 01:30:02.793075 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:02.793046    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-40f90a-29818170-bzz4w"
Sep 11 01:30:02.797037 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:02.797015    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818170-wl98m"
Sep 11 01:30:02.853478 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:02.853451    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b9930f-29818170-5gflq"
Sep 11 01:30:03.631098 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:03.631066    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818170-wl98m"
Sep 11 01:30:03.631298 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:03.631099    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818170-wl98m" event={"ID":"bbab7a05-c5ce-4039-a867-24421fd51d48","Type":"ContainerDied","Data":"38853ab8e5da34fd40d47315d4def91f4a620f3491bb509f1db15d49a138211d"}
Sep 11 01:30:03.631298 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:03.631149    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="38853ab8e5da34fd40d47315d4def91f4a620f3491bb509f1db15d49a138211d"
Sep 11 01:30:03.632439 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:03.632422    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-40f90a-29818170-bzz4w"
Sep 11 01:30:03.632587 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:03.632465    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-40f90a-29818170-bzz4w" event={"ID":"ab95ce39-5e6c-40b4-8b64-cfe7c01fb401","Type":"ContainerDied","Data":"28b3a981d730363301b2891075853e099b4c9b74955b2731d15c16fa64dd0f54"}
Sep 11 01:30:03.632587 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:03.632507    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="28b3a981d730363301b2891075853e099b4c9b74955b2731d15c16fa64dd0f54"
Sep 11 01:30:03.634069 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:03.634050    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b9930f-29818170-5gflq" event={"ID":"8795cde5-6a3a-4201-bb24-c0d3bfe1c1a8","Type":"ContainerDied","Data":"680e9ae240652c9226c3bc28aa498f485244aaa8c2ba13d0200fef42133bd30c"}
Sep 11 01:30:03.634167 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:03.634083    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="680e9ae240652c9226c3bc28aa498f485244aaa8c2ba13d0200fef42133bd30c"
Sep 11 01:30:03.634167 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:03.634083    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b9930f-29818170-5gflq"
Sep 11 01:30:05.622030 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:05.621998    2574 scope.go:117] "RemoveContainer" containerID="8ad371b337d271db08173b17b09c2f47315b988a56546908ad55219b4677293a"
Sep 11 01:30:09.808164 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:09.808130    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:30:15.240529 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:15.240500    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-40f90a-6b7f8c5df9-fpjwc_99909fa4-b502-45ba-a93e-433e11679741/payload-processing/0.log"
Sep 11 01:30:24.510271 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:24.510238    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:30:48.174632 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:48.174599    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-c1edc6-7c6f5584df-f5pkt_bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b/payload-processing/0.log"
Sep 11 01:30:48.420955 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:48.420902    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-40f90a-6b7f8c5df9-fpjwc_99909fa4-b502-45ba-a93e-433e11679741/payload-processing/0.log"
Sep 11 01:30:48.697209 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:48.697168    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:30:49.014100 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:49.014021    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:30:49.474379 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:49.474345    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:30:51.942552 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:51.942507    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:30:52.035856 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:52.035788    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:30:53.256662 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:53.256623    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:30:53.357450 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:53.357422    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:30:53.614327 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:53.614248    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:30:53.717251 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:53.717219    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:30:53.818601 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:53.818569    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:30:53.851730 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:53.851698    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8fa57c-6fb987f4d7-6kvq6"]
Sep 11 01:30:53.852107 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:53.852093    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8795cde5-6a3a-4201-bb24-c0d3bfe1c1a8" containerName="cleanup"
Sep 11 01:30:53.852170 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:53.852109    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8795cde5-6a3a-4201-bb24-c0d3bfe1c1a8" containerName="cleanup"
Sep 11 01:30:53.852170 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:53.852121    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbab7a05-c5ce-4039-a867-24421fd51d48" containerName="cleanup"
Sep 11 01:30:53.852170 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:53.852126    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbab7a05-c5ce-4039-a867-24421fd51d48" containerName="cleanup"
Sep 11 01:30:53.852170 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:53.852133    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab95ce39-5e6c-40b4-8b64-cfe7c01fb401" containerName="cleanup"
Sep 11 01:30:53.852170 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:53.852139    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab95ce39-5e6c-40b4-8b64-cfe7c01fb401" containerName="cleanup"
Sep 11 01:30:53.852323 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:53.852191    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab95ce39-5e6c-40b4-8b64-cfe7c01fb401" containerName="cleanup"
Sep 11 01:30:53.852323 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:53.852198    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8795cde5-6a3a-4201-bb24-c0d3bfe1c1a8" containerName="cleanup"
Sep 11 01:30:53.852323 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:53.852205    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbab7a05-c5ce-4039-a867-24421fd51d48" containerName="cleanup"
Sep 11 01:30:53.855454 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:53.855431    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8fa57c-6fb987f4d7-6kvq6"
Sep 11 01:30:53.858302 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:53.858278    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-8fa57c\""
Sep 11 01:30:53.858445 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:53.858303    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-8fa57c-dockercfg-gwlws\""
Sep 11 01:30:53.866236 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:53.866177    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8fa57c-6fb987f4d7-6kvq6"]
Sep 11 01:30:53.974531 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:53.974497    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ad88d8c-dba3-4a8e-851b-2f1d4c950372-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-8fa57c-6fb987f4d7-6kvq6\" (UID: \"7ad88d8c-dba3-4a8e-851b-2f1d4c950372\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8fa57c-6fb987f4d7-6kvq6"
Sep 11 01:30:53.974684 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:53.974561    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzncz\" (UniqueName: \"kubernetes.io/projected/7ad88d8c-dba3-4a8e-851b-2f1d4c950372-kube-api-access-bzncz\") pod \"payload-processing-e2e-ipp-cleanup-8fa57c-6fb987f4d7-6kvq6\" (UID: \"7ad88d8c-dba3-4a8e-851b-2f1d4c950372\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8fa57c-6fb987f4d7-6kvq6"
Sep 11 01:30:54.075610 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:54.075577    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzncz\" (UniqueName: \"kubernetes.io/projected/7ad88d8c-dba3-4a8e-851b-2f1d4c950372-kube-api-access-bzncz\") pod \"payload-processing-e2e-ipp-cleanup-8fa57c-6fb987f4d7-6kvq6\" (UID: \"7ad88d8c-dba3-4a8e-851b-2f1d4c950372\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8fa57c-6fb987f4d7-6kvq6"
Sep 11 01:30:54.075767 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:54.075637    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ad88d8c-dba3-4a8e-851b-2f1d4c950372-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-8fa57c-6fb987f4d7-6kvq6\" (UID: \"7ad88d8c-dba3-4a8e-851b-2f1d4c950372\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8fa57c-6fb987f4d7-6kvq6"
Sep 11 01:30:54.076373 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:54.076354    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ad88d8c-dba3-4a8e-851b-2f1d4c950372-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-8fa57c-6fb987f4d7-6kvq6\" (UID: \"7ad88d8c-dba3-4a8e-851b-2f1d4c950372\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8fa57c-6fb987f4d7-6kvq6"
Sep 11 01:30:54.083060 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:54.083039    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzncz\" (UniqueName: \"kubernetes.io/projected/7ad88d8c-dba3-4a8e-851b-2f1d4c950372-kube-api-access-bzncz\") pod \"payload-processing-e2e-ipp-cleanup-8fa57c-6fb987f4d7-6kvq6\" (UID: \"7ad88d8c-dba3-4a8e-851b-2f1d4c950372\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8fa57c-6fb987f4d7-6kvq6"
Sep 11 01:30:54.167863 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:54.167839    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8fa57c-6fb987f4d7-6kvq6"
Sep 11 01:30:54.439728 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:54.439655    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8fa57c-6fb987f4d7-6kvq6"]
Sep 11 01:30:54.442681 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:30:54.442654    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ad88d8c_dba3_4a8e_851b_2f1d4c950372.slice/crio-98f403dad9963b798ce1d0f493f6e257892ceda4e7e3c44b8f309fff01214014 WatchSource:0}: Error finding container 98f403dad9963b798ce1d0f493f6e257892ceda4e7e3c44b8f309fff01214014: Status 404 returned error can't find the container with id 98f403dad9963b798ce1d0f493f6e257892ceda4e7e3c44b8f309fff01214014
Sep 11 01:30:54.891609 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:54.891523    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8fa57c-6fb987f4d7-6kvq6" event={"ID":"7ad88d8c-dba3-4a8e-851b-2f1d4c950372","Type":"ContainerStarted","Data":"b69c48b0f7edd1ffaa96016ca97f9820d5d1537aa2e82ba7855004debca39b19"}
Sep 11 01:30:54.891609 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:54.891578    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8fa57c-6fb987f4d7-6kvq6" event={"ID":"7ad88d8c-dba3-4a8e-851b-2f1d4c950372","Type":"ContainerStarted","Data":"98f403dad9963b798ce1d0f493f6e257892ceda4e7e3c44b8f309fff01214014"}
Sep 11 01:30:54.891794 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:54.891647    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8fa57c-6fb987f4d7-6kvq6"
Sep 11 01:30:54.907657 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:30:54.907606    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8fa57c-6fb987f4d7-6kvq6" podStartSLOduration=1.9075906360000001 podStartE2EDuration="1.907590636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:30:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:30:54.905272777 +0000 UTC m=+1970.302464950" watchObservedRunningTime="2026-09-11 01:30:54.907590636 +0000 UTC m=+1970.304782808"
Sep 11 01:31:05.899023 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:05.898936    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8fa57c-6fb987f4d7-6kvq6"
Sep 11 01:31:16.991642 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:16.991576    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:31:17.697713 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:17.697682    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8fa57c-gd5jm"]
Sep 11 01:31:17.701085 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:17.701069    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8fa57c-gd5jm"
Sep 11 01:31:17.704272 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:17.704251    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q6gbn\""
Sep 11 01:31:17.704367 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:17.704275    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 01:31:17.710539 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:17.710468    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8fa57c-gd5jm"]
Sep 11 01:31:17.788626 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:17.788591    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8393245a-19a3-4518-8e96-79a2807141ba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-8fa57c-gd5jm\" (UID: \"8393245a-19a3-4518-8e96-79a2807141ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8fa57c-gd5jm"
Sep 11 01:31:17.890090 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:17.890056    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8393245a-19a3-4518-8e96-79a2807141ba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-8fa57c-gd5jm\" (UID: \"8393245a-19a3-4518-8e96-79a2807141ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8fa57c-gd5jm"
Sep 11 01:31:17.890691 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:17.890669    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8393245a-19a3-4518-8e96-79a2807141ba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-8fa57c-gd5jm\" (UID: \"8393245a-19a3-4518-8e96-79a2807141ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8fa57c-gd5jm"
Sep 11 01:31:18.013077 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:18.012987    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8fa57c-gd5jm"
Sep 11 01:31:18.281583 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:18.281555    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8fa57c-gd5jm"]
Sep 11 01:31:18.283754 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:31:18.283729    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8393245a_19a3_4518_8e96_79a2807141ba.slice/crio-e03adf85654cc5faf967876eb8a662e036d0596a30515aac0d00f2454b10f648 WatchSource:0}: Error finding container e03adf85654cc5faf967876eb8a662e036d0596a30515aac0d00f2454b10f648: Status 404 returned error can't find the container with id e03adf85654cc5faf967876eb8a662e036d0596a30515aac0d00f2454b10f648
Sep 11 01:31:19.013178 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:19.013144    2574 generic.go:356] "Generic (PLEG): container finished" podID="8393245a-19a3-4518-8e96-79a2807141ba" containerID="ea50b35ac07009106f1e00ed116ab5386c3093f3906e387c7631eb220e139229" exitCode=0
Sep 11 01:31:19.013590 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:19.013229    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8fa57c-gd5jm" event={"ID":"8393245a-19a3-4518-8e96-79a2807141ba","Type":"ContainerDied","Data":"ea50b35ac07009106f1e00ed116ab5386c3093f3906e387c7631eb220e139229"}
Sep 11 01:31:19.013590 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:19.013298    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8fa57c-gd5jm" event={"ID":"8393245a-19a3-4518-8e96-79a2807141ba","Type":"ContainerStarted","Data":"e03adf85654cc5faf967876eb8a662e036d0596a30515aac0d00f2454b10f648"}
Sep 11 01:31:20.156874 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:20.156848    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8fa57c-gd5jm"
Sep 11 01:31:20.314307 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:20.314194    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8393245a-19a3-4518-8e96-79a2807141ba-maas-api-service-ca\") pod \"8393245a-19a3-4518-8e96-79a2807141ba\" (UID: \"8393245a-19a3-4518-8e96-79a2807141ba\") "
Sep 11 01:31:20.314631 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:20.314603    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8393245a-19a3-4518-8e96-79a2807141ba-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8393245a-19a3-4518-8e96-79a2807141ba" (UID: "8393245a-19a3-4518-8e96-79a2807141ba"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:31:20.415538 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:20.415496    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8393245a-19a3-4518-8e96-79a2807141ba-maas-api-service-ca\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:31:21.025612 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:21.025579    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8fa57c-gd5jm"
Sep 11 01:31:21.025810 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:21.025577    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8fa57c-gd5jm" event={"ID":"8393245a-19a3-4518-8e96-79a2807141ba","Type":"ContainerDied","Data":"e03adf85654cc5faf967876eb8a662e036d0596a30515aac0d00f2454b10f648"}
Sep 11 01:31:21.025810 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:21.025723    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e03adf85654cc5faf967876eb8a662e036d0596a30515aac0d00f2454b10f648"
Sep 11 01:31:23.822523 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:23.822492    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:31:23.850528 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:23.850495    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8fa57c-6fb987f4d7-6kvq6"]
Sep 11 01:31:23.850764 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:23.850732    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8fa57c-6fb987f4d7-6kvq6" podUID="7ad88d8c-dba3-4a8e-851b-2f1d4c950372" containerName="payload-processing" containerID="cri-o://b69c48b0f7edd1ffaa96016ca97f9820d5d1537aa2e82ba7855004debca39b19" gracePeriod=30
Sep 11 01:31:23.892944 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:23.892901    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:31:24.001180 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:24.001141    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:31:24.043382 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:24.043350    2574 generic.go:356] "Generic (PLEG): container finished" podID="7ad88d8c-dba3-4a8e-851b-2f1d4c950372" containerID="b69c48b0f7edd1ffaa96016ca97f9820d5d1537aa2e82ba7855004debca39b19" exitCode=0
Sep 11 01:31:24.043576 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:24.043407    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8fa57c-6fb987f4d7-6kvq6" event={"ID":"7ad88d8c-dba3-4a8e-851b-2f1d4c950372","Type":"ContainerDied","Data":"b69c48b0f7edd1ffaa96016ca97f9820d5d1537aa2e82ba7855004debca39b19"}
Sep 11 01:31:24.085369 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:24.085309    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8fa57c-gd5jm"]
Sep 11 01:31:24.087275 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:24.087252    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8fa57c-gd5jm"]
Sep 11 01:31:24.094088 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:24.094056    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:31:24.097276 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:24.097258    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8fa57c-6fb987f4d7-6kvq6"
Sep 11 01:31:24.147366 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:24.147336    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzncz\" (UniqueName: \"kubernetes.io/projected/7ad88d8c-dba3-4a8e-851b-2f1d4c950372-kube-api-access-bzncz\") pod \"7ad88d8c-dba3-4a8e-851b-2f1d4c950372\" (UID: \"7ad88d8c-dba3-4a8e-851b-2f1d4c950372\") "
Sep 11 01:31:24.147500 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:24.147386    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ad88d8c-dba3-4a8e-851b-2f1d4c950372-plugins-config-volume\") pod \"7ad88d8c-dba3-4a8e-851b-2f1d4c950372\" (UID: \"7ad88d8c-dba3-4a8e-851b-2f1d4c950372\") "
Sep 11 01:31:24.147776 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:24.147755    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ad88d8c-dba3-4a8e-851b-2f1d4c950372-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7ad88d8c-dba3-4a8e-851b-2f1d4c950372" (UID: "7ad88d8c-dba3-4a8e-851b-2f1d4c950372"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:31:24.149528 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:24.149509    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ad88d8c-dba3-4a8e-851b-2f1d4c950372-kube-api-access-bzncz" (OuterVolumeSpecName: "kube-api-access-bzncz") pod "7ad88d8c-dba3-4a8e-851b-2f1d4c950372" (UID: "7ad88d8c-dba3-4a8e-851b-2f1d4c950372"). InnerVolumeSpecName "kube-api-access-bzncz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:31:24.154399 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:24.154374    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:31:24.248832 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:24.248807    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ad88d8c-dba3-4a8e-851b-2f1d4c950372-plugins-config-volume\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:31:24.248832 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:24.248831    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzncz\" (UniqueName: \"kubernetes.io/projected/7ad88d8c-dba3-4a8e-851b-2f1d4c950372-kube-api-access-bzncz\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:31:24.458509 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:24.458474    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:31:25.049262 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:25.049230    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8fa57c-6fb987f4d7-6kvq6" event={"ID":"7ad88d8c-dba3-4a8e-851b-2f1d4c950372","Type":"ContainerDied","Data":"98f403dad9963b798ce1d0f493f6e257892ceda4e7e3c44b8f309fff01214014"}
Sep 11 01:31:25.049700 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:25.049284    2574 scope.go:117] "RemoveContainer" containerID="b69c48b0f7edd1ffaa96016ca97f9820d5d1537aa2e82ba7855004debca39b19"
Sep 11 01:31:25.049700 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:25.049241    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8fa57c-6fb987f4d7-6kvq6"
Sep 11 01:31:25.069353 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:25.069327    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8fa57c-6fb987f4d7-6kvq6"]
Sep 11 01:31:25.073088 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:25.073066    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8fa57c-6fb987f4d7-6kvq6"]
Sep 11 01:31:25.099613 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:25.099585    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ad88d8c-dba3-4a8e-851b-2f1d4c950372" path="/var/lib/kubelet/pods/7ad88d8c-dba3-4a8e-851b-2f1d4c950372/volumes"
Sep 11 01:31:25.099932 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:25.099898    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8393245a-19a3-4518-8e96-79a2807141ba" path="/var/lib/kubelet/pods/8393245a-19a3-4518-8e96-79a2807141ba/volumes"
Sep 11 01:31:30.397994 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:30.397960    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:31:30.456458 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:30.456430    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:31:30.689659 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:30.689627    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:31:34.530199 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:34.530162    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c1edc6-g5wtr"]
Sep 11 01:31:34.530591 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:34.530562    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ad88d8c-dba3-4a8e-851b-2f1d4c950372" containerName="payload-processing"
Sep 11 01:31:34.530591 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:34.530575    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ad88d8c-dba3-4a8e-851b-2f1d4c950372" containerName="payload-processing"
Sep 11 01:31:34.530591 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:34.530583    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8393245a-19a3-4518-8e96-79a2807141ba" containerName="revoke-keys"
Sep 11 01:31:34.530591 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:34.530589    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8393245a-19a3-4518-8e96-79a2807141ba" containerName="revoke-keys"
Sep 11 01:31:34.530748 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:34.530659    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8393245a-19a3-4518-8e96-79a2807141ba" containerName="revoke-keys"
Sep 11 01:31:34.530748 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:34.530668    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ad88d8c-dba3-4a8e-851b-2f1d4c950372" containerName="payload-processing"
Sep 11 01:31:34.535104 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:34.535088    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c1edc6-g5wtr"
Sep 11 01:31:34.538874 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:34.538850    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 01:31:34.539003 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:34.538850    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q6gbn\""
Sep 11 01:31:34.547295 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:34.547272    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c1edc6-g5wtr"]
Sep 11 01:31:34.636594 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:34.636571    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d432977-cf75-40a4-b84b-eecf8e4a7fe7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-c1edc6-g5wtr\" (UID: \"4d432977-cf75-40a4-b84b-eecf8e4a7fe7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c1edc6-g5wtr"
Sep 11 01:31:34.640701 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:34.640663    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:31:34.737501 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:34.737476    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d432977-cf75-40a4-b84b-eecf8e4a7fe7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-c1edc6-g5wtr\" (UID: \"4d432977-cf75-40a4-b84b-eecf8e4a7fe7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c1edc6-g5wtr"
Sep 11 01:31:34.738063 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:34.738046    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d432977-cf75-40a4-b84b-eecf8e4a7fe7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-c1edc6-g5wtr\" (UID: \"4d432977-cf75-40a4-b84b-eecf8e4a7fe7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c1edc6-g5wtr"
Sep 11 01:31:34.845445 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:34.845382    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c1edc6-g5wtr"
Sep 11 01:31:35.119043 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:35.118984    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c1edc6-g5wtr"]
Sep 11 01:31:35.121330 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:31:35.121297    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d432977_cf75_40a4_b84b_eecf8e4a7fe7.slice/crio-1474466763df33247d844218b3dcda8719b933c8cdda4df7d4b83e27060fbccf WatchSource:0}: Error finding container 1474466763df33247d844218b3dcda8719b933c8cdda4df7d4b83e27060fbccf: Status 404 returned error can't find the container with id 1474466763df33247d844218b3dcda8719b933c8cdda4df7d4b83e27060fbccf
Sep 11 01:31:36.107974 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:36.107872    2574 generic.go:356] "Generic (PLEG): container finished" podID="4d432977-cf75-40a4-b84b-eecf8e4a7fe7" containerID="fbd7561a281a9141f7e47f1fa5d820c779a8bc7cdd7d3160c4ac3d27c449f7fc" exitCode=0
Sep 11 01:31:36.107974 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:36.107955    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c1edc6-g5wtr" event={"ID":"4d432977-cf75-40a4-b84b-eecf8e4a7fe7","Type":"ContainerDied","Data":"fbd7561a281a9141f7e47f1fa5d820c779a8bc7cdd7d3160c4ac3d27c449f7fc"}
Sep 11 01:31:36.108350 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:36.108004    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c1edc6-g5wtr" event={"ID":"4d432977-cf75-40a4-b84b-eecf8e4a7fe7","Type":"ContainerStarted","Data":"1474466763df33247d844218b3dcda8719b933c8cdda4df7d4b83e27060fbccf"}
Sep 11 01:31:37.244333 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:37.244308    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c1edc6-g5wtr"
Sep 11 01:31:37.359147 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:37.359123    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d432977-cf75-40a4-b84b-eecf8e4a7fe7-maas-api-service-ca\") pod \"4d432977-cf75-40a4-b84b-eecf8e4a7fe7\" (UID: \"4d432977-cf75-40a4-b84b-eecf8e4a7fe7\") "
Sep 11 01:31:37.359457 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:37.359436    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d432977-cf75-40a4-b84b-eecf8e4a7fe7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4d432977-cf75-40a4-b84b-eecf8e4a7fe7" (UID: "4d432977-cf75-40a4-b84b-eecf8e4a7fe7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:31:37.460350 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:37.460320    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d432977-cf75-40a4-b84b-eecf8e4a7fe7-maas-api-service-ca\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:31:38.118598 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:38.118569    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c1edc6-g5wtr"
Sep 11 01:31:38.118598 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:38.118586    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c1edc6-g5wtr" event={"ID":"4d432977-cf75-40a4-b84b-eecf8e4a7fe7","Type":"ContainerDied","Data":"1474466763df33247d844218b3dcda8719b933c8cdda4df7d4b83e27060fbccf"}
Sep 11 01:31:38.118808 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:38.118629    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1474466763df33247d844218b3dcda8719b933c8cdda4df7d4b83e27060fbccf"
Sep 11 01:31:50.034018 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:50.033987    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:31:50.056375 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:50.056338    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c1edc6-7c6f5584df-f5pkt"]
Sep 11 01:31:50.056588 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:50.056565    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-c1edc6-7c6f5584df-f5pkt" podUID="bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b" containerName="payload-processing" containerID="cri-o://869480a80310c9925abb59ad03497bc9ade6fbb3383ab49a249b87f5035b7790" gracePeriod=30
Sep 11 01:31:50.064681 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:50.064653    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c1edc6-86d99bdbf8-8hhms"]
Sep 11 01:31:50.064893 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:50.064859    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c1edc6-86d99bdbf8-8hhms" podUID="7e8ddd83-87bd-443c-b068-e3bff6af6654" containerName="payload-pre-processing" containerID="cri-o://1d3961b92797337a24f5ed11d3bfa7b68c1865c68159fec84d6e654a25462821" gracePeriod=30
Sep 11 01:31:50.098088 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:50.098043    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:31:50.181854 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:50.181793    2574 generic.go:356] "Generic (PLEG): container finished" podID="bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b" containerID="869480a80310c9925abb59ad03497bc9ade6fbb3383ab49a249b87f5035b7790" exitCode=0
Sep 11 01:31:50.181991 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:50.181858    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-c1edc6-7c6f5584df-f5pkt" event={"ID":"bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b","Type":"ContainerDied","Data":"869480a80310c9925abb59ad03497bc9ade6fbb3383ab49a249b87f5035b7790"}
Sep 11 01:31:50.197869 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:50.197827    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:31:50.224969 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:50.224936    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c1edc6-g5wtr"]
Sep 11 01:31:50.228065 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:50.228036    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c1edc6-g5wtr"]
Sep 11 01:31:50.256873 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:50.256838    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:31:50.343265 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:50.343242    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-c1edc6-7c6f5584df-f5pkt"
Sep 11 01:31:50.346169 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:50.346149    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c1edc6-86d99bdbf8-8hhms"
Sep 11 01:31:50.484746 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:50.484707    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b-plugins-config-volume\") pod \"bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b\" (UID: \"bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b\") "
Sep 11 01:31:50.484968 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:50.484779    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e8ddd83-87bd-443c-b068-e3bff6af6654-plugins-config-volume\") pod \"7e8ddd83-87bd-443c-b068-e3bff6af6654\" (UID: \"7e8ddd83-87bd-443c-b068-e3bff6af6654\") "
Sep 11 01:31:50.484968 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:50.484809    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrz6j\" (UniqueName: \"kubernetes.io/projected/7e8ddd83-87bd-443c-b068-e3bff6af6654-kube-api-access-lrz6j\") pod \"7e8ddd83-87bd-443c-b068-e3bff6af6654\" (UID: \"7e8ddd83-87bd-443c-b068-e3bff6af6654\") "
Sep 11 01:31:50.484968 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:50.484937    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r2dwb\" (UniqueName: \"kubernetes.io/projected/bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b-kube-api-access-r2dwb\") pod \"bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b\" (UID: \"bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b\") "
Sep 11 01:31:50.485185 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:50.485154    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b" (UID: "bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:31:50.485253 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:50.485230    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e8ddd83-87bd-443c-b068-e3bff6af6654-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7e8ddd83-87bd-443c-b068-e3bff6af6654" (UID: "7e8ddd83-87bd-443c-b068-e3bff6af6654"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:31:50.487211 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:50.487191    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b-kube-api-access-r2dwb" (OuterVolumeSpecName: "kube-api-access-r2dwb") pod "bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b" (UID: "bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b"). InnerVolumeSpecName "kube-api-access-r2dwb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:31:50.487211 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:50.487198    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e8ddd83-87bd-443c-b068-e3bff6af6654-kube-api-access-lrz6j" (OuterVolumeSpecName: "kube-api-access-lrz6j") pod "7e8ddd83-87bd-443c-b068-e3bff6af6654" (UID: "7e8ddd83-87bd-443c-b068-e3bff6af6654"). InnerVolumeSpecName "kube-api-access-lrz6j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:31:50.585721 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:50.585626    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2dwb\" (UniqueName: \"kubernetes.io/projected/bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b-kube-api-access-r2dwb\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:31:50.585721 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:50.585659    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b-plugins-config-volume\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:31:50.585721 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:50.585674    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e8ddd83-87bd-443c-b068-e3bff6af6654-plugins-config-volume\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:31:50.585721 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:50.585685    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrz6j\" (UniqueName: \"kubernetes.io/projected/7e8ddd83-87bd-443c-b068-e3bff6af6654-kube-api-access-lrz6j\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:31:50.625833 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:50.625795    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:31:51.100363 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:51.100324    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d432977-cf75-40a4-b84b-eecf8e4a7fe7" path="/var/lib/kubelet/pods/4d432977-cf75-40a4-b84b-eecf8e4a7fe7/volumes"
Sep 11 01:31:51.105970 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:51.105939    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:31:51.189248 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:51.189214    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-c1edc6-7c6f5584df-f5pkt"
Sep 11 01:31:51.189446 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:51.189253    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-c1edc6-7c6f5584df-f5pkt" event={"ID":"bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b","Type":"ContainerDied","Data":"09367f14dd0d9af08fe57fc16cbe860ea0073f6079bdbfb24525accb80970fa5"}
Sep 11 01:31:51.189446 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:51.189318    2574 scope.go:117] "RemoveContainer" containerID="869480a80310c9925abb59ad03497bc9ade6fbb3383ab49a249b87f5035b7790"
Sep 11 01:31:51.190885 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:51.190579    2574 generic.go:356] "Generic (PLEG): container finished" podID="7e8ddd83-87bd-443c-b068-e3bff6af6654" containerID="1d3961b92797337a24f5ed11d3bfa7b68c1865c68159fec84d6e654a25462821" exitCode=0
Sep 11 01:31:51.190885 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:51.190617    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c1edc6-86d99bdbf8-8hhms"
Sep 11 01:31:51.190885 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:51.190667    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c1edc6-86d99bdbf8-8hhms" event={"ID":"7e8ddd83-87bd-443c-b068-e3bff6af6654","Type":"ContainerDied","Data":"1d3961b92797337a24f5ed11d3bfa7b68c1865c68159fec84d6e654a25462821"}
Sep 11 01:31:51.190885 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:51.190728    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c1edc6-86d99bdbf8-8hhms" event={"ID":"7e8ddd83-87bd-443c-b068-e3bff6af6654","Type":"ContainerDied","Data":"ce16c9743ae3b9d0ba5266c0f8ce5ab5070073bbd2b280c2e6f920faa6e05a82"}
Sep 11 01:31:51.205229 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:51.205208    2574 scope.go:117] "RemoveContainer" containerID="1d3961b92797337a24f5ed11d3bfa7b68c1865c68159fec84d6e654a25462821"
Sep 11 01:31:51.214164 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:51.214137    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c1edc6-86d99bdbf8-8hhms"]
Sep 11 01:31:51.214421 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:51.214396    2574 scope.go:117] "RemoveContainer" containerID="1d3961b92797337a24f5ed11d3bfa7b68c1865c68159fec84d6e654a25462821"
Sep 11 01:31:51.214766 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:31:51.214731    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d3961b92797337a24f5ed11d3bfa7b68c1865c68159fec84d6e654a25462821\": container with ID starting with 1d3961b92797337a24f5ed11d3bfa7b68c1865c68159fec84d6e654a25462821 not found: ID does not exist" containerID="1d3961b92797337a24f5ed11d3bfa7b68c1865c68159fec84d6e654a25462821"
Sep 11 01:31:51.214876 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:51.214777    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d3961b92797337a24f5ed11d3bfa7b68c1865c68159fec84d6e654a25462821"} err="failed to get container status \"1d3961b92797337a24f5ed11d3bfa7b68c1865c68159fec84d6e654a25462821\": rpc error: code = NotFound desc = could not find container \"1d3961b92797337a24f5ed11d3bfa7b68c1865c68159fec84d6e654a25462821\": container with ID starting with 1d3961b92797337a24f5ed11d3bfa7b68c1865c68159fec84d6e654a25462821 not found: ID does not exist"
Sep 11 01:31:51.217236 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:51.217211    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c1edc6-86d99bdbf8-8hhms"]
Sep 11 01:31:51.226336 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:51.226311    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c1edc6-7c6f5584df-f5pkt"]
Sep 11 01:31:51.230665 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:51.230640    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c1edc6-7c6f5584df-f5pkt"]
Sep 11 01:31:53.100506 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:53.100465    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e8ddd83-87bd-443c-b068-e3bff6af6654" path="/var/lib/kubelet/pods/7e8ddd83-87bd-443c-b068-e3bff6af6654/volumes"
Sep 11 01:31:53.101049 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:53.101028    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b" path="/var/lib/kubelet/pods/bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b/volumes"
Sep 11 01:31:55.061202 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:55.061166    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:31:55.649572 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:31:55.649535    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:32:05.653418 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:05.653381    2574 scope.go:117] "RemoveContainer" containerID="4f7ad6ddf73195d7fdcb12be117dedea4c59d76584a5cbc05ffef1ec0ac3da57"
Sep 11 01:32:05.661559 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:05.661539    2574 scope.go:117] "RemoveContainer" containerID="6053c1ab13feeb80d53f7e094a57497b8aa5bce45a881da67a6e5685989e0e99"
Sep 11 01:32:25.805663 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:25.805629    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:32:32.418693 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:32.418591    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:32:32.914240 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:32.914204    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:32:38.567787 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:38.567754    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:32:38.679465 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:38.679432    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:32:38.907608 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:38.907527    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:32:39.411460 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:39.411427    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:32:49.905562 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:49.905528    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-40f90a-29818170-bzz4w"]
Sep 11 01:32:49.906899 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:49.906866    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:32:49.910275 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:49.910249    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-40f90a-29818170-bzz4w"]
Sep 11 01:32:49.929030 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:49.929000    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-40f90a-6b7f8c5df9-fpjwc"]
Sep 11 01:32:49.929270 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:49.929244    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-40f90a-6b7f8c5df9-fpjwc" podUID="99909fa4-b502-45ba-a93e-433e11679741" containerName="payload-processing" containerID="cri-o://2ac08ce3e6a9e9b949be6a10f8b646af3f62c4cf286c33610a5eb8be96fc904f" gracePeriod=30
Sep 11 01:32:49.935260 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:49.935233    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-40f90a-58cb6f886b-xmwmb"]
Sep 11 01:32:49.935453 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:49.935431    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-40f90a-58cb6f886b-xmwmb" podUID="9932705f-5b73-4290-b6d0-6fbef66a72d0" containerName="payload-pre-processing" containerID="cri-o://599d1005a5678ed3c83e679c81542fd613afd409daa609953f6e50de12ab7524" gracePeriod=30
Sep 11 01:32:49.975933 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:49.975875    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:32:50.041686 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.041658    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:32:50.102651 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.102617    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:32:50.193437 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.193412    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-40f90a-6b7f8c5df9-fpjwc"
Sep 11 01:32:50.216506 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.216481    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-40f90a-58cb6f886b-xmwmb"
Sep 11 01:32:50.218399 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.218376    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7zwdq\" (UniqueName: \"kubernetes.io/projected/99909fa4-b502-45ba-a93e-433e11679741-kube-api-access-7zwdq\") pod \"99909fa4-b502-45ba-a93e-433e11679741\" (UID: \"99909fa4-b502-45ba-a93e-433e11679741\") "
Sep 11 01:32:50.218535 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.218420    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nlkh9\" (UniqueName: \"kubernetes.io/projected/9932705f-5b73-4290-b6d0-6fbef66a72d0-kube-api-access-nlkh9\") pod \"9932705f-5b73-4290-b6d0-6fbef66a72d0\" (UID: \"9932705f-5b73-4290-b6d0-6fbef66a72d0\") "
Sep 11 01:32:50.218535 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.218512    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99909fa4-b502-45ba-a93e-433e11679741-plugins-config-volume\") pod \"99909fa4-b502-45ba-a93e-433e11679741\" (UID: \"99909fa4-b502-45ba-a93e-433e11679741\") "
Sep 11 01:32:50.218657 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.218541    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9932705f-5b73-4290-b6d0-6fbef66a72d0-plugins-config-volume\") pod \"9932705f-5b73-4290-b6d0-6fbef66a72d0\" (UID: \"9932705f-5b73-4290-b6d0-6fbef66a72d0\") "
Sep 11 01:32:50.218967 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.218939    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99909fa4-b502-45ba-a93e-433e11679741-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "99909fa4-b502-45ba-a93e-433e11679741" (UID: "99909fa4-b502-45ba-a93e-433e11679741"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:32:50.219248 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.219040    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9932705f-5b73-4290-b6d0-6fbef66a72d0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9932705f-5b73-4290-b6d0-6fbef66a72d0" (UID: "9932705f-5b73-4290-b6d0-6fbef66a72d0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:32:50.220718 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.220690    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9932705f-5b73-4290-b6d0-6fbef66a72d0-kube-api-access-nlkh9" (OuterVolumeSpecName: "kube-api-access-nlkh9") pod "9932705f-5b73-4290-b6d0-6fbef66a72d0" (UID: "9932705f-5b73-4290-b6d0-6fbef66a72d0"). InnerVolumeSpecName "kube-api-access-nlkh9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:32:50.220810 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.220694    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99909fa4-b502-45ba-a93e-433e11679741-kube-api-access-7zwdq" (OuterVolumeSpecName: "kube-api-access-7zwdq") pod "99909fa4-b502-45ba-a93e-433e11679741" (UID: "99909fa4-b502-45ba-a93e-433e11679741"). InnerVolumeSpecName "kube-api-access-7zwdq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:32:50.319570 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.319535    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99909fa4-b502-45ba-a93e-433e11679741-plugins-config-volume\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:32:50.319570 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.319564    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9932705f-5b73-4290-b6d0-6fbef66a72d0-plugins-config-volume\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:32:50.319570 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.319573    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zwdq\" (UniqueName: \"kubernetes.io/projected/99909fa4-b502-45ba-a93e-433e11679741-kube-api-access-7zwdq\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:32:50.319570 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.319581    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nlkh9\" (UniqueName: \"kubernetes.io/projected/9932705f-5b73-4290-b6d0-6fbef66a72d0-kube-api-access-nlkh9\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:32:50.498439 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.498338    2574 generic.go:356] "Generic (PLEG): container finished" podID="99909fa4-b502-45ba-a93e-433e11679741" containerID="2ac08ce3e6a9e9b949be6a10f8b646af3f62c4cf286c33610a5eb8be96fc904f" exitCode=0
Sep 11 01:32:50.498439 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.498383    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-40f90a-6b7f8c5df9-fpjwc"
Sep 11 01:32:50.498439 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.498417    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-40f90a-6b7f8c5df9-fpjwc" event={"ID":"99909fa4-b502-45ba-a93e-433e11679741","Type":"ContainerDied","Data":"2ac08ce3e6a9e9b949be6a10f8b646af3f62c4cf286c33610a5eb8be96fc904f"}
Sep 11 01:32:50.498711 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.498475    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-40f90a-6b7f8c5df9-fpjwc" event={"ID":"99909fa4-b502-45ba-a93e-433e11679741","Type":"ContainerDied","Data":"16afc56b6f36a6ce2a315da2faee221e5ce6ce3ef58dcf26123cae2e41e751b7"}
Sep 11 01:32:50.498711 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.498503    2574 scope.go:117] "RemoveContainer" containerID="2ac08ce3e6a9e9b949be6a10f8b646af3f62c4cf286c33610a5eb8be96fc904f"
Sep 11 01:32:50.499925 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.499888    2574 generic.go:356] "Generic (PLEG): container finished" podID="9932705f-5b73-4290-b6d0-6fbef66a72d0" containerID="599d1005a5678ed3c83e679c81542fd613afd409daa609953f6e50de12ab7524" exitCode=0
Sep 11 01:32:50.500029 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.499944    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-40f90a-58cb6f886b-xmwmb"
Sep 11 01:32:50.500029 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.499939    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-40f90a-58cb6f886b-xmwmb" event={"ID":"9932705f-5b73-4290-b6d0-6fbef66a72d0","Type":"ContainerDied","Data":"599d1005a5678ed3c83e679c81542fd613afd409daa609953f6e50de12ab7524"}
Sep 11 01:32:50.500029 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.500009    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-40f90a-58cb6f886b-xmwmb" event={"ID":"9932705f-5b73-4290-b6d0-6fbef66a72d0","Type":"ContainerDied","Data":"4d83191cb90a577c238daf790fba3a3be74122453acc606cf46ac5d15e34dec1"}
Sep 11 01:32:50.508478 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.508460    2574 scope.go:117] "RemoveContainer" containerID="2ac08ce3e6a9e9b949be6a10f8b646af3f62c4cf286c33610a5eb8be96fc904f"
Sep 11 01:32:50.508752 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:32:50.508731    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ac08ce3e6a9e9b949be6a10f8b646af3f62c4cf286c33610a5eb8be96fc904f\": container with ID starting with 2ac08ce3e6a9e9b949be6a10f8b646af3f62c4cf286c33610a5eb8be96fc904f not found: ID does not exist" containerID="2ac08ce3e6a9e9b949be6a10f8b646af3f62c4cf286c33610a5eb8be96fc904f"
Sep 11 01:32:50.508800 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.508761    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ac08ce3e6a9e9b949be6a10f8b646af3f62c4cf286c33610a5eb8be96fc904f"} err="failed to get container status \"2ac08ce3e6a9e9b949be6a10f8b646af3f62c4cf286c33610a5eb8be96fc904f\": rpc error: code = NotFound desc = could not find container \"2ac08ce3e6a9e9b949be6a10f8b646af3f62c4cf286c33610a5eb8be96fc904f\": container with ID starting with 2ac08ce3e6a9e9b949be6a10f8b646af3f62c4cf286c33610a5eb8be96fc904f not found: ID does not exist"
Sep 11 01:32:50.508800 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.508778    2574 scope.go:117] "RemoveContainer" containerID="599d1005a5678ed3c83e679c81542fd613afd409daa609953f6e50de12ab7524"
Sep 11 01:32:50.518570 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.518553    2574 scope.go:117] "RemoveContainer" containerID="599d1005a5678ed3c83e679c81542fd613afd409daa609953f6e50de12ab7524"
Sep 11 01:32:50.518851 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:32:50.518834    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"599d1005a5678ed3c83e679c81542fd613afd409daa609953f6e50de12ab7524\": container with ID starting with 599d1005a5678ed3c83e679c81542fd613afd409daa609953f6e50de12ab7524 not found: ID does not exist" containerID="599d1005a5678ed3c83e679c81542fd613afd409daa609953f6e50de12ab7524"
Sep 11 01:32:50.518898 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.518856    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"599d1005a5678ed3c83e679c81542fd613afd409daa609953f6e50de12ab7524"} err="failed to get container status \"599d1005a5678ed3c83e679c81542fd613afd409daa609953f6e50de12ab7524\": rpc error: code = NotFound desc = could not find container \"599d1005a5678ed3c83e679c81542fd613afd409daa609953f6e50de12ab7524\": container with ID starting with 599d1005a5678ed3c83e679c81542fd613afd409daa609953f6e50de12ab7524 not found: ID does not exist"
Sep 11 01:32:50.522108 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.522087    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-40f90a-58cb6f886b-xmwmb"]
Sep 11 01:32:50.526087 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.526065    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-40f90a-58cb6f886b-xmwmb"]
Sep 11 01:32:50.535633 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.535614    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-40f90a-6b7f8c5df9-fpjwc"]
Sep 11 01:32:50.539324 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.539304    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-40f90a-6b7f8c5df9-fpjwc"]
Sep 11 01:32:50.541756 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:50.541737    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:32:51.099719 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:51.099686    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9932705f-5b73-4290-b6d0-6fbef66a72d0" path="/var/lib/kubelet/pods/9932705f-5b73-4290-b6d0-6fbef66a72d0/volumes"
Sep 11 01:32:51.100112 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:51.100058    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99909fa4-b502-45ba-a93e-433e11679741" path="/var/lib/kubelet/pods/99909fa4-b502-45ba-a93e-433e11679741/volumes"
Sep 11 01:32:51.100294 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:51.100279    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab95ce39-5e6c-40b4-8b64-cfe7c01fb401" path="/var/lib/kubelet/pods/ab95ce39-5e6c-40b4-8b64-cfe7c01fb401/volumes"
Sep 11 01:32:56.222853 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:56.222823    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:32:56.322503 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:56.322472    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:32:56.507844 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:56.507757    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:32:56.898410 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:56.898333    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-d6e26b-sh72l"]
Sep 11 01:32:56.898719 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:56.898705    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e8ddd83-87bd-443c-b068-e3bff6af6654" containerName="payload-pre-processing"
Sep 11 01:32:56.898769 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:56.898722    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e8ddd83-87bd-443c-b068-e3bff6af6654" containerName="payload-pre-processing"
Sep 11 01:32:56.898769 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:56.898731    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9932705f-5b73-4290-b6d0-6fbef66a72d0" containerName="payload-pre-processing"
Sep 11 01:32:56.898769 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:56.898736    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9932705f-5b73-4290-b6d0-6fbef66a72d0" containerName="payload-pre-processing"
Sep 11 01:32:56.898769 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:56.898747    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99909fa4-b502-45ba-a93e-433e11679741" containerName="payload-processing"
Sep 11 01:32:56.898769 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:56.898753    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="99909fa4-b502-45ba-a93e-433e11679741" containerName="payload-processing"
Sep 11 01:32:56.898769 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:56.898762    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b" containerName="payload-processing"
Sep 11 01:32:56.898769 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:56.898767    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b" containerName="payload-processing"
Sep 11 01:32:56.899051 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:56.898783    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d432977-cf75-40a4-b84b-eecf8e4a7fe7" containerName="revoke-keys"
Sep 11 01:32:56.899051 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:56.898788    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d432977-cf75-40a4-b84b-eecf8e4a7fe7" containerName="revoke-keys"
Sep 11 01:32:56.899051 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:56.898839    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9932705f-5b73-4290-b6d0-6fbef66a72d0" containerName="payload-pre-processing"
Sep 11 01:32:56.899051 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:56.898845    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e8ddd83-87bd-443c-b068-e3bff6af6654" containerName="payload-pre-processing"
Sep 11 01:32:56.899051 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:56.898853    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="99909fa4-b502-45ba-a93e-433e11679741" containerName="payload-processing"
Sep 11 01:32:56.899051 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:56.898860    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="bed1b6f2-b6ad-42d4-bd46-1e3e7509cf1b" containerName="payload-processing"
Sep 11 01:32:56.899051 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:56.898867    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d432977-cf75-40a4-b84b-eecf8e4a7fe7" containerName="revoke-keys"
Sep 11 01:32:56.903131 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:56.903113    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-d6e26b-sh72l"
Sep 11 01:32:56.905387 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:56.905369    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 01:32:56.905455 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:56.905369    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q6gbn\""
Sep 11 01:32:56.909715 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:56.909688    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-d6e26b-sh72l"]
Sep 11 01:32:56.978076 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:56.978047    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65f50dae-7c6c-40e9-812d-74e38bce6533-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-d6e26b-sh72l\" (UID: \"65f50dae-7c6c-40e9-812d-74e38bce6533\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-d6e26b-sh72l"
Sep 11 01:32:57.079015 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:57.078984    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65f50dae-7c6c-40e9-812d-74e38bce6533-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-d6e26b-sh72l\" (UID: \"65f50dae-7c6c-40e9-812d-74e38bce6533\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-d6e26b-sh72l"
Sep 11 01:32:57.079588 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:57.079568    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65f50dae-7c6c-40e9-812d-74e38bce6533-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-d6e26b-sh72l\" (UID: \"65f50dae-7c6c-40e9-812d-74e38bce6533\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-d6e26b-sh72l"
Sep 11 01:32:57.115683 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:57.115650    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:32:57.214750 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:57.214722    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-d6e26b-sh72l"
Sep 11 01:32:57.486550 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:57.486517    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-d6e26b-sh72l"]
Sep 11 01:32:57.487800 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:32:57.487772    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65f50dae_7c6c_40e9_812d_74e38bce6533.slice/crio-b9566b21890f979839ed66cefec74a35c0deea322e39f810f05f647c1710d6e3 WatchSource:0}: Error finding container b9566b21890f979839ed66cefec74a35c0deea322e39f810f05f647c1710d6e3: Status 404 returned error can't find the container with id b9566b21890f979839ed66cefec74a35c0deea322e39f810f05f647c1710d6e3
Sep 11 01:32:57.535816 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:57.535783    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-d6e26b-sh72l" event={"ID":"65f50dae-7c6c-40e9-812d-74e38bce6533","Type":"ContainerStarted","Data":"b9566b21890f979839ed66cefec74a35c0deea322e39f810f05f647c1710d6e3"}
Sep 11 01:32:58.543080 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:58.542980    2574 generic.go:356] "Generic (PLEG): container finished" podID="65f50dae-7c6c-40e9-812d-74e38bce6533" containerID="5971acf7bd8f958a78169b8720cd9ec753ac8d351da6a1cb2012632e87a7f3be" exitCode=0
Sep 11 01:32:58.543454 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:58.543076    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-d6e26b-sh72l" event={"ID":"65f50dae-7c6c-40e9-812d-74e38bce6533","Type":"ContainerDied","Data":"5971acf7bd8f958a78169b8720cd9ec753ac8d351da6a1cb2012632e87a7f3be"}
Sep 11 01:32:59.676837 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:59.676811    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-d6e26b-sh72l"
Sep 11 01:32:59.700943 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:59.700921    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65f50dae-7c6c-40e9-812d-74e38bce6533-maas-api-service-ca\") pod \"65f50dae-7c6c-40e9-812d-74e38bce6533\" (UID: \"65f50dae-7c6c-40e9-812d-74e38bce6533\") "
Sep 11 01:32:59.701212 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:59.701192    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65f50dae-7c6c-40e9-812d-74e38bce6533-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "65f50dae-7c6c-40e9-812d-74e38bce6533" (UID: "65f50dae-7c6c-40e9-812d-74e38bce6533"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:32:59.801734 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:32:59.801646    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65f50dae-7c6c-40e9-812d-74e38bce6533-maas-api-service-ca\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:33:00.554093 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:00.554061    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-d6e26b-sh72l"
Sep 11 01:33:00.554093 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:00.554079    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-d6e26b-sh72l" event={"ID":"65f50dae-7c6c-40e9-812d-74e38bce6533","Type":"ContainerDied","Data":"b9566b21890f979839ed66cefec74a35c0deea322e39f810f05f647c1710d6e3"}
Sep 11 01:33:00.554301 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:00.554146    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b9566b21890f979839ed66cefec74a35c0deea322e39f810f05f647c1710d6e3"
Sep 11 01:33:05.694151 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:05.694119    2574 scope.go:117] "RemoveContainer" containerID="4d29509998f3b41f6b0e4da6f616b989b54f2ca988b0a1434658e2dce7595ee4"
Sep 11 01:33:05.702242 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:05.702225    2574 scope.go:117] "RemoveContainer" containerID="232af026be7c68d431c5b653d299c7b9251a721905e32f11a26ee159845b1912"
Sep 11 01:33:05.780850 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:05.780824    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:33:07.421556 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:07.421513    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:33:07.443819 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:07.443787    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-d6e26b-78d4d6b489-jq8ct"]
Sep 11 01:33:07.444072 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:07.444041    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d6e26b-78d4d6b489-jq8ct" podUID="047af868-ffeb-4356-9ed0-3b87b0a548d5" containerName="payload-processing" containerID="cri-o://aa827dc00c170b65b0d4e242b1d41d4687fbf5975be15cc32c4a3c3b971f20ff" gracePeriod=30
Sep 11 01:33:07.454804 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:07.454774    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d6e26b-64f4dc94fd-ckm8g"]
Sep 11 01:33:07.455049 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:07.455024    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d6e26b-64f4dc94fd-ckm8g" podUID="5f7b932a-27a8-465e-881e-9bb78f4c508b" containerName="payload-pre-processing" containerID="cri-o://00fad2f1502d58b5ca8abdf34535f6c36e6c8aeb8ff98d80d53abf198999a538" gracePeriod=30
Sep 11 01:33:07.517438 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:07.517401    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:33:07.591831 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:07.591800    2574 generic.go:356] "Generic (PLEG): container finished" podID="047af868-ffeb-4356-9ed0-3b87b0a548d5" containerID="aa827dc00c170b65b0d4e242b1d41d4687fbf5975be15cc32c4a3c3b971f20ff" exitCode=0
Sep 11 01:33:07.592022 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:07.591882    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d6e26b-78d4d6b489-jq8ct" event={"ID":"047af868-ffeb-4356-9ed0-3b87b0a548d5","Type":"ContainerDied","Data":"aa827dc00c170b65b0d4e242b1d41d4687fbf5975be15cc32c4a3c3b971f20ff"}
Sep 11 01:33:07.593695 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:07.593675    2574 generic.go:356] "Generic (PLEG): container finished" podID="5f7b932a-27a8-465e-881e-9bb78f4c508b" containerID="00fad2f1502d58b5ca8abdf34535f6c36e6c8aeb8ff98d80d53abf198999a538" exitCode=0
Sep 11 01:33:07.593832 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:07.593711    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d6e26b-64f4dc94fd-ckm8g" event={"ID":"5f7b932a-27a8-465e-881e-9bb78f4c508b","Type":"ContainerDied","Data":"00fad2f1502d58b5ca8abdf34535f6c36e6c8aeb8ff98d80d53abf198999a538"}
Sep 11 01:33:07.607779 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:07.607749    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-d6e26b-sh72l"]
Sep 11 01:33:07.609663 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:07.609625    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-d6e26b-sh72l"]
Sep 11 01:33:07.610177 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:07.610153    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:33:07.674157 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:07.674095    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:33:07.699753 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:07.699730    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d6e26b-78d4d6b489-jq8ct"
Sep 11 01:33:07.730559 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:07.730536    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d6e26b-64f4dc94fd-ckm8g"
Sep 11 01:33:07.773416 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:07.773391    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f7b932a-27a8-465e-881e-9bb78f4c508b-plugins-config-volume\") pod \"5f7b932a-27a8-465e-881e-9bb78f4c508b\" (UID: \"5f7b932a-27a8-465e-881e-9bb78f4c508b\") "
Sep 11 01:33:07.773525 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:07.773467    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rw4rx\" (UniqueName: \"kubernetes.io/projected/5f7b932a-27a8-465e-881e-9bb78f4c508b-kube-api-access-rw4rx\") pod \"5f7b932a-27a8-465e-881e-9bb78f4c508b\" (UID: \"5f7b932a-27a8-465e-881e-9bb78f4c508b\") "
Sep 11 01:33:07.773576 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:07.773530    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/047af868-ffeb-4356-9ed0-3b87b0a548d5-plugins-config-volume\") pod \"047af868-ffeb-4356-9ed0-3b87b0a548d5\" (UID: \"047af868-ffeb-4356-9ed0-3b87b0a548d5\") "
Sep 11 01:33:07.773617 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:07.773577    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6frvp\" (UniqueName: \"kubernetes.io/projected/047af868-ffeb-4356-9ed0-3b87b0a548d5-kube-api-access-6frvp\") pod \"047af868-ffeb-4356-9ed0-3b87b0a548d5\" (UID: \"047af868-ffeb-4356-9ed0-3b87b0a548d5\") "
Sep 11 01:33:07.773890 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:07.773865    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f7b932a-27a8-465e-881e-9bb78f4c508b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5f7b932a-27a8-465e-881e-9bb78f4c508b" (UID: "5f7b932a-27a8-465e-881e-9bb78f4c508b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:33:07.773990 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:07.773891    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/047af868-ffeb-4356-9ed0-3b87b0a548d5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "047af868-ffeb-4356-9ed0-3b87b0a548d5" (UID: "047af868-ffeb-4356-9ed0-3b87b0a548d5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:33:07.775613 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:07.775587    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f7b932a-27a8-465e-881e-9bb78f4c508b-kube-api-access-rw4rx" (OuterVolumeSpecName: "kube-api-access-rw4rx") pod "5f7b932a-27a8-465e-881e-9bb78f4c508b" (UID: "5f7b932a-27a8-465e-881e-9bb78f4c508b"). InnerVolumeSpecName "kube-api-access-rw4rx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:33:07.775740 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:07.775724    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/047af868-ffeb-4356-9ed0-3b87b0a548d5-kube-api-access-6frvp" (OuterVolumeSpecName: "kube-api-access-6frvp") pod "047af868-ffeb-4356-9ed0-3b87b0a548d5" (UID: "047af868-ffeb-4356-9ed0-3b87b0a548d5"). InnerVolumeSpecName "kube-api-access-6frvp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:33:07.874665 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:07.874637    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/047af868-ffeb-4356-9ed0-3b87b0a548d5-plugins-config-volume\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:33:07.874665 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:07.874661    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6frvp\" (UniqueName: \"kubernetes.io/projected/047af868-ffeb-4356-9ed0-3b87b0a548d5-kube-api-access-6frvp\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:33:07.874824 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:07.874671    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f7b932a-27a8-465e-881e-9bb78f4c508b-plugins-config-volume\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:33:07.874824 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:07.874681    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rw4rx\" (UniqueName: \"kubernetes.io/projected/5f7b932a-27a8-465e-881e-9bb78f4c508b-kube-api-access-rw4rx\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:33:08.010841 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:08.010780    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:33:08.599291 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:08.599261    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d6e26b-78d4d6b489-jq8ct"
Sep 11 01:33:08.599702 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:08.599261    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-d6e26b-78d4d6b489-jq8ct" event={"ID":"047af868-ffeb-4356-9ed0-3b87b0a548d5","Type":"ContainerDied","Data":"3a65bab7c32036cb29637e440f1428d370a9f20f322b8c2b9edec0c10351ecfc"}
Sep 11 01:33:08.599702 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:08.599402    2574 scope.go:117] "RemoveContainer" containerID="aa827dc00c170b65b0d4e242b1d41d4687fbf5975be15cc32c4a3c3b971f20ff"
Sep 11 01:33:08.600671 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:08.600649    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d6e26b-64f4dc94fd-ckm8g"
Sep 11 01:33:08.600797 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:08.600687    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d6e26b-64f4dc94fd-ckm8g" event={"ID":"5f7b932a-27a8-465e-881e-9bb78f4c508b","Type":"ContainerDied","Data":"cafe742d1b5cff95b5a9c096e48e1d94aef9ef16ffca8e92cf8535f04e97a452"}
Sep 11 01:33:08.610265 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:08.610243    2574 scope.go:117] "RemoveContainer" containerID="00fad2f1502d58b5ca8abdf34535f6c36e6c8aeb8ff98d80d53abf198999a538"
Sep 11 01:33:08.623782 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:08.623761    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-d6e26b-78d4d6b489-jq8ct"]
Sep 11 01:33:08.627386 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:08.627366    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-d6e26b-78d4d6b489-jq8ct"]
Sep 11 01:33:08.641536 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:08.641515    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d6e26b-64f4dc94fd-ckm8g"]
Sep 11 01:33:08.646227 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:08.646206    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-d6e26b-64f4dc94fd-ckm8g"]
Sep 11 01:33:09.099902 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:09.099868    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="047af868-ffeb-4356-9ed0-3b87b0a548d5" path="/var/lib/kubelet/pods/047af868-ffeb-4356-9ed0-3b87b0a548d5/volumes"
Sep 11 01:33:09.100255 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:09.100239    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f7b932a-27a8-465e-881e-9bb78f4c508b" path="/var/lib/kubelet/pods/5f7b932a-27a8-465e-881e-9bb78f4c508b/volumes"
Sep 11 01:33:09.100525 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:09.100514    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65f50dae-7c6c-40e9-812d-74e38bce6533" path="/var/lib/kubelet/pods/65f50dae-7c6c-40e9-812d-74e38bce6533/volumes"
Sep 11 01:33:14.019247 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:14.019211    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:33:14.117585 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:14.117553    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:33:14.310988 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:14.310881    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:33:14.695221 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:14.695185    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-b9930f-7mcbp"]
Sep 11 01:33:14.695576 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:14.695557    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f7b932a-27a8-465e-881e-9bb78f4c508b" containerName="payload-pre-processing"
Sep 11 01:33:14.695654 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:14.695578    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f7b932a-27a8-465e-881e-9bb78f4c508b" containerName="payload-pre-processing"
Sep 11 01:33:14.695654 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:14.695595    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65f50dae-7c6c-40e9-812d-74e38bce6533" containerName="revoke-keys"
Sep 11 01:33:14.695654 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:14.695604    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="65f50dae-7c6c-40e9-812d-74e38bce6533" containerName="revoke-keys"
Sep 11 01:33:14.695654 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:14.695620    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="047af868-ffeb-4356-9ed0-3b87b0a548d5" containerName="payload-processing"
Sep 11 01:33:14.695654 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:14.695628    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="047af868-ffeb-4356-9ed0-3b87b0a548d5" containerName="payload-processing"
Sep 11 01:33:14.695952 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:14.695721    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f7b932a-27a8-465e-881e-9bb78f4c508b" containerName="payload-pre-processing"
Sep 11 01:33:14.695952 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:14.695735    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="65f50dae-7c6c-40e9-812d-74e38bce6533" containerName="revoke-keys"
Sep 11 01:33:14.695952 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:14.695746    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="047af868-ffeb-4356-9ed0-3b87b0a548d5" containerName="payload-processing"
Sep 11 01:33:14.699120 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:14.699100    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-b9930f-7mcbp"
Sep 11 01:33:14.701320 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:14.701295    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 01:33:14.701432 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:14.701362    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q6gbn\""
Sep 11 01:33:14.704580 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:14.704554    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-b9930f-7mcbp"]
Sep 11 01:33:14.820892 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:14.820867    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:33:14.836131 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:14.836105    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9108447-0c6d-4d10-814d-d7cb625d76f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-b9930f-7mcbp\" (UID: \"d9108447-0c6d-4d10-814d-d7cb625d76f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-b9930f-7mcbp"
Sep 11 01:33:14.936687 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:14.936660    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9108447-0c6d-4d10-814d-d7cb625d76f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-b9930f-7mcbp\" (UID: \"d9108447-0c6d-4d10-814d-d7cb625d76f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-b9930f-7mcbp"
Sep 11 01:33:14.937276 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:14.937257    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9108447-0c6d-4d10-814d-d7cb625d76f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-b9930f-7mcbp\" (UID: \"d9108447-0c6d-4d10-814d-d7cb625d76f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-b9930f-7mcbp"
Sep 11 01:33:15.010528 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:15.010448    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-b9930f-7mcbp"
Sep 11 01:33:15.282214 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:15.282127    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-b9930f-7mcbp"]
Sep 11 01:33:15.284505 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:33:15.284474    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9108447_0c6d_4d10_814d_d7cb625d76f6.slice/crio-15ecb8351b9d44554bfcdfb438a0e7ce0562bf9339bf1726b9c7448cd0a7a40e WatchSource:0}: Error finding container 15ecb8351b9d44554bfcdfb438a0e7ce0562bf9339bf1726b9c7448cd0a7a40e: Status 404 returned error can't find the container with id 15ecb8351b9d44554bfcdfb438a0e7ce0562bf9339bf1726b9c7448cd0a7a40e
Sep 11 01:33:15.638160 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:15.638081    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-b9930f-7mcbp" event={"ID":"d9108447-0c6d-4d10-814d-d7cb625d76f6","Type":"ContainerStarted","Data":"6344b5899888a15530084ac6d6edf56ecf6b8ce13aec6bc8b1cabcd21a99a40f"}
Sep 11 01:33:15.638160 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:15.638132    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-b9930f-7mcbp" event={"ID":"d9108447-0c6d-4d10-814d-d7cb625d76f6","Type":"ContainerStarted","Data":"15ecb8351b9d44554bfcdfb438a0e7ce0562bf9339bf1726b9c7448cd0a7a40e"}
Sep 11 01:33:15.652481 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:15.652432    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-b9930f-7mcbp" podStartSLOduration=1.652416658 podStartE2EDuration="1.652416658s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:33:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:33:15.651027418 +0000 UTC m=+2111.048219591" watchObservedRunningTime="2026-09-11 01:33:15.652416658 +0000 UTC m=+2111.049608831"
Sep 11 01:33:16.644264 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:16.644228    2574 generic.go:356] "Generic (PLEG): container finished" podID="d9108447-0c6d-4d10-814d-d7cb625d76f6" containerID="6344b5899888a15530084ac6d6edf56ecf6b8ce13aec6bc8b1cabcd21a99a40f" exitCode=0
Sep 11 01:33:16.644617 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:16.644308    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-b9930f-7mcbp" event={"ID":"d9108447-0c6d-4d10-814d-d7cb625d76f6","Type":"ContainerDied","Data":"6344b5899888a15530084ac6d6edf56ecf6b8ce13aec6bc8b1cabcd21a99a40f"}
Sep 11 01:33:17.781543 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:17.781521    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-b9930f-7mcbp"
Sep 11 01:33:17.862203 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:17.862175    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9108447-0c6d-4d10-814d-d7cb625d76f6-maas-api-service-ca\") pod \"d9108447-0c6d-4d10-814d-d7cb625d76f6\" (UID: \"d9108447-0c6d-4d10-814d-d7cb625d76f6\") "
Sep 11 01:33:17.862539 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:17.862516    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9108447-0c6d-4d10-814d-d7cb625d76f6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d9108447-0c6d-4d10-814d-d7cb625d76f6" (UID: "d9108447-0c6d-4d10-814d-d7cb625d76f6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:33:17.963393 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:17.963365    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9108447-0c6d-4d10-814d-d7cb625d76f6-maas-api-service-ca\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:33:18.655492 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:18.655458    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-b9930f-7mcbp"
Sep 11 01:33:18.655661 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:18.655508    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-b9930f-7mcbp" event={"ID":"d9108447-0c6d-4d10-814d-d7cb625d76f6","Type":"ContainerDied","Data":"15ecb8351b9d44554bfcdfb438a0e7ce0562bf9339bf1726b9c7448cd0a7a40e"}
Sep 11 01:33:18.655661 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:18.655555    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="15ecb8351b9d44554bfcdfb438a0e7ce0562bf9339bf1726b9c7448cd0a7a40e"
Sep 11 01:33:25.156787 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:25.156752    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:33:25.160587 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:25.160557    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b9930f-29818170-5gflq"]
Sep 11 01:33:25.162854 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:25.162832    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b9930f-29818170-5gflq"]
Sep 11 01:33:25.194247 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:25.194219    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b9930f-cb59bb98b-p7ndh"]
Sep 11 01:33:25.194496 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:25.194472    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b9930f-cb59bb98b-p7ndh" podUID="d5a9e347-6003-4aa2-8025-ffc8823483d0" containerName="payload-pre-processing" containerID="cri-o://f7868a051bbb73f57d6724a9d763b81cabbb4040fa7d585f2038caa1a7f10725" gracePeriod=30
Sep 11 01:33:25.258440 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:25.258399    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:33:25.347955 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:25.347878    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-b9930f-7mcbp"]
Sep 11 01:33:25.351271 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:25.351245    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:33:25.351742 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:25.351713    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-b9930f-7mcbp"]
Sep 11 01:33:25.413824 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:25.413801    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:33:25.446282 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:25.446255    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b9930f-cb59bb98b-p7ndh"
Sep 11 01:33:25.534241 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:25.534209    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5a9e347-6003-4aa2-8025-ffc8823483d0-plugins-config-volume\") pod \"d5a9e347-6003-4aa2-8025-ffc8823483d0\" (UID: \"d5a9e347-6003-4aa2-8025-ffc8823483d0\") "
Sep 11 01:33:25.534241 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:25.534245    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p276m\" (UniqueName: \"kubernetes.io/projected/d5a9e347-6003-4aa2-8025-ffc8823483d0-kube-api-access-p276m\") pod \"d5a9e347-6003-4aa2-8025-ffc8823483d0\" (UID: \"d5a9e347-6003-4aa2-8025-ffc8823483d0\") "
Sep 11 01:33:25.534576 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:25.534554    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5a9e347-6003-4aa2-8025-ffc8823483d0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d5a9e347-6003-4aa2-8025-ffc8823483d0" (UID: "d5a9e347-6003-4aa2-8025-ffc8823483d0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:33:25.536493 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:25.536475    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5a9e347-6003-4aa2-8025-ffc8823483d0-kube-api-access-p276m" (OuterVolumeSpecName: "kube-api-access-p276m") pod "d5a9e347-6003-4aa2-8025-ffc8823483d0" (UID: "d5a9e347-6003-4aa2-8025-ffc8823483d0"). InnerVolumeSpecName "kube-api-access-p276m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:33:25.634925 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:25.634881    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5a9e347-6003-4aa2-8025-ffc8823483d0-plugins-config-volume\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:33:25.634925 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:25.634924    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p276m\" (UniqueName: \"kubernetes.io/projected/d5a9e347-6003-4aa2-8025-ffc8823483d0-kube-api-access-p276m\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:33:25.688355 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:25.688287    2574 generic.go:356] "Generic (PLEG): container finished" podID="d5a9e347-6003-4aa2-8025-ffc8823483d0" containerID="f7868a051bbb73f57d6724a9d763b81cabbb4040fa7d585f2038caa1a7f10725" exitCode=0
Sep 11 01:33:25.688355 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:25.688331    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b9930f-cb59bb98b-p7ndh"
Sep 11 01:33:25.688481 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:25.688366    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b9930f-cb59bb98b-p7ndh" event={"ID":"d5a9e347-6003-4aa2-8025-ffc8823483d0","Type":"ContainerDied","Data":"f7868a051bbb73f57d6724a9d763b81cabbb4040fa7d585f2038caa1a7f10725"}
Sep 11 01:33:25.688481 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:25.688418    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b9930f-cb59bb98b-p7ndh" event={"ID":"d5a9e347-6003-4aa2-8025-ffc8823483d0","Type":"ContainerDied","Data":"8dc46c7744dd43bc0f7d756254f62915b6536e157f8ef4760e5990f20259ecb5"}
Sep 11 01:33:25.688481 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:25.688444    2574 scope.go:117] "RemoveContainer" containerID="f7868a051bbb73f57d6724a9d763b81cabbb4040fa7d585f2038caa1a7f10725"
Sep 11 01:33:25.697511 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:25.697495    2574 scope.go:117] "RemoveContainer" containerID="f7868a051bbb73f57d6724a9d763b81cabbb4040fa7d585f2038caa1a7f10725"
Sep 11 01:33:25.697790 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:33:25.697768    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7868a051bbb73f57d6724a9d763b81cabbb4040fa7d585f2038caa1a7f10725\": container with ID starting with f7868a051bbb73f57d6724a9d763b81cabbb4040fa7d585f2038caa1a7f10725 not found: ID does not exist" containerID="f7868a051bbb73f57d6724a9d763b81cabbb4040fa7d585f2038caa1a7f10725"
Sep 11 01:33:25.697844 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:25.697802    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7868a051bbb73f57d6724a9d763b81cabbb4040fa7d585f2038caa1a7f10725"} err="failed to get container status \"f7868a051bbb73f57d6724a9d763b81cabbb4040fa7d585f2038caa1a7f10725\": rpc error: code = NotFound desc = could not find container \"f7868a051bbb73f57d6724a9d763b81cabbb4040fa7d585f2038caa1a7f10725\": container with ID starting with f7868a051bbb73f57d6724a9d763b81cabbb4040fa7d585f2038caa1a7f10725 not found: ID does not exist"
Sep 11 01:33:25.708460 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:25.708439    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b9930f-cb59bb98b-p7ndh"]
Sep 11 01:33:25.710473 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:25.710452    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b9930f-cb59bb98b-p7ndh"]
Sep 11 01:33:25.758389 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:25.758366    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:33:27.101543 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:27.101507    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8795cde5-6a3a-4201-bb24-c0d3bfe1c1a8" path="/var/lib/kubelet/pods/8795cde5-6a3a-4201-bb24-c0d3bfe1c1a8/volumes"
Sep 11 01:33:27.101971 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:27.101957    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5a9e347-6003-4aa2-8025-ffc8823483d0" path="/var/lib/kubelet/pods/d5a9e347-6003-4aa2-8025-ffc8823483d0/volumes"
Sep 11 01:33:27.102288 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:27.102274    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9108447-0c6d-4d10-814d-d7cb625d76f6" path="/var/lib/kubelet/pods/d9108447-0c6d-4d10-814d-d7cb625d76f6/volumes"
Sep 11 01:33:31.766752 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:31.766718    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:33:31.838848 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:31.838824    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:33:32.102488 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:33:32.102398    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:34:58.801182 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:34:58.801137    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:35:00.816421 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:35:00.816387    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:35:02.231887 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:35:02.231845    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:36:05.755398 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:36:05.755361    2574 scope.go:117] "RemoveContainer" containerID="5e518a1391d20ec5bc8b4b1d3042720661d51f1100a82332d7cf916f6a61f63a"
Sep 11 01:36:05.763608 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:36:05.763588    2574 scope.go:117] "RemoveContainer" containerID="1203bb437eadcbe7a38baea86bd0ef45318c1625b4db1d9cebc97f9f5a6f1990"
Sep 11 01:36:58.185073 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:36:58.184962    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:37:20.155827 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:37:20.155791    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:37:20.256025 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:37:20.255983    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:38:00.283348 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:00.283314    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w"]
Sep 11 01:38:00.283983 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:00.283647    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5a9e347-6003-4aa2-8025-ffc8823483d0" containerName="payload-pre-processing"
Sep 11 01:38:00.283983 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:00.283658    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5a9e347-6003-4aa2-8025-ffc8823483d0" containerName="payload-pre-processing"
Sep 11 01:38:00.283983 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:00.283676    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9108447-0c6d-4d10-814d-d7cb625d76f6" containerName="revoke-keys"
Sep 11 01:38:00.283983 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:00.283681    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9108447-0c6d-4d10-814d-d7cb625d76f6" containerName="revoke-keys"
Sep 11 01:38:00.283983 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:00.283733    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5a9e347-6003-4aa2-8025-ffc8823483d0" containerName="payload-pre-processing"
Sep 11 01:38:00.283983 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:00.283744    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9108447-0c6d-4d10-814d-d7cb625d76f6" containerName="revoke-keys"
Sep 11 01:38:00.286806 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:00.286780    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w"
Sep 11 01:38:00.289039 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:00.289019    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-d3e6934d-kserve-self-signed-certs\""
Sep 11 01:38:00.296114 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:00.296091    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w"]
Sep 11 01:38:00.313351 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:00.313318    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:38:00.366027 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:00.366002    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a2fa686-72be-4b87-b779-87c62131c59b-dshm\") pod \"e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w\" (UID: \"5a2fa686-72be-4b87-b779-87c62131c59b\") " pod="llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w"
Sep 11 01:38:00.366140 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:00.366053    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a2fa686-72be-4b87-b779-87c62131c59b-tmp-dir\") pod \"e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w\" (UID: \"5a2fa686-72be-4b87-b779-87c62131c59b\") " pod="llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w"
Sep 11 01:38:00.366140 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:00.366113    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85brc\" (UniqueName: \"kubernetes.io/projected/5a2fa686-72be-4b87-b779-87c62131c59b-kube-api-access-85brc\") pod \"e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w\" (UID: \"5a2fa686-72be-4b87-b779-87c62131c59b\") " pod="llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w"
Sep 11 01:38:00.366252 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:00.366137    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a2fa686-72be-4b87-b779-87c62131c59b-home\") pod \"e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w\" (UID: \"5a2fa686-72be-4b87-b779-87c62131c59b\") " pod="llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w"
Sep 11 01:38:00.366252 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:00.366178    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a2fa686-72be-4b87-b779-87c62131c59b-model-cache\") pod \"e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w\" (UID: \"5a2fa686-72be-4b87-b779-87c62131c59b\") " pod="llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w"
Sep 11 01:38:00.366320 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:00.366252    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a2fa686-72be-4b87-b779-87c62131c59b-tls-certs\") pod \"e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w\" (UID: \"5a2fa686-72be-4b87-b779-87c62131c59b\") " pod="llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w"
Sep 11 01:38:00.466943 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:00.466900    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a2fa686-72be-4b87-b779-87c62131c59b-tls-certs\") pod \"e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w\" (UID: \"5a2fa686-72be-4b87-b779-87c62131c59b\") " pod="llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w"
Sep 11 01:38:00.467046 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:00.466962    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a2fa686-72be-4b87-b779-87c62131c59b-dshm\") pod \"e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w\" (UID: \"5a2fa686-72be-4b87-b779-87c62131c59b\") " pod="llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w"
Sep 11 01:38:00.467046 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:00.466991    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a2fa686-72be-4b87-b779-87c62131c59b-tmp-dir\") pod \"e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w\" (UID: \"5a2fa686-72be-4b87-b779-87c62131c59b\") " pod="llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w"
Sep 11 01:38:00.467046 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:00.467014    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85brc\" (UniqueName: \"kubernetes.io/projected/5a2fa686-72be-4b87-b779-87c62131c59b-kube-api-access-85brc\") pod \"e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w\" (UID: \"5a2fa686-72be-4b87-b779-87c62131c59b\") " pod="llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w"
Sep 11 01:38:00.467046 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:00.467029    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a2fa686-72be-4b87-b779-87c62131c59b-home\") pod \"e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w\" (UID: \"5a2fa686-72be-4b87-b779-87c62131c59b\") " pod="llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w"
Sep 11 01:38:00.467245 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:00.467048    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a2fa686-72be-4b87-b779-87c62131c59b-model-cache\") pod \"e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w\" (UID: \"5a2fa686-72be-4b87-b779-87c62131c59b\") " pod="llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w"
Sep 11 01:38:00.467397 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:00.467372    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a2fa686-72be-4b87-b779-87c62131c59b-model-cache\") pod \"e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w\" (UID: \"5a2fa686-72be-4b87-b779-87c62131c59b\") " pod="llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w"
Sep 11 01:38:00.467500 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:00.467445    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a2fa686-72be-4b87-b779-87c62131c59b-home\") pod \"e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w\" (UID: \"5a2fa686-72be-4b87-b779-87c62131c59b\") " pod="llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w"
Sep 11 01:38:00.467540 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:00.467504    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a2fa686-72be-4b87-b779-87c62131c59b-tmp-dir\") pod \"e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w\" (UID: \"5a2fa686-72be-4b87-b779-87c62131c59b\") " pod="llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w"
Sep 11 01:38:00.469414 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:00.469386    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a2fa686-72be-4b87-b779-87c62131c59b-dshm\") pod \"e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w\" (UID: \"5a2fa686-72be-4b87-b779-87c62131c59b\") " pod="llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w"
Sep 11 01:38:00.469584 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:00.469566    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a2fa686-72be-4b87-b779-87c62131c59b-tls-certs\") pod \"e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w\" (UID: \"5a2fa686-72be-4b87-b779-87c62131c59b\") " pod="llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w"
Sep 11 01:38:00.474140 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:00.474121    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85brc\" (UniqueName: \"kubernetes.io/projected/5a2fa686-72be-4b87-b779-87c62131c59b-kube-api-access-85brc\") pod \"e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w\" (UID: \"5a2fa686-72be-4b87-b779-87c62131c59b\") " pod="llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w"
Sep 11 01:38:00.596694 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:00.596620    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w"
Sep 11 01:38:00.867825 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:00.867797    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w"]
Sep 11 01:38:00.870338 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:38:00.870313    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5a2fa686_72be_4b87_b779_87c62131c59b.slice/crio-2a6dd2f60516c113b72e1d2751b8514a1b92e529e31dc75c34d86f787e5de4ff WatchSource:0}: Error finding container 2a6dd2f60516c113b72e1d2751b8514a1b92e529e31dc75c34d86f787e5de4ff: Status 404 returned error can't find the container with id 2a6dd2f60516c113b72e1d2751b8514a1b92e529e31dc75c34d86f787e5de4ff
Sep 11 01:38:01.004185 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:01.004151    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w" event={"ID":"5a2fa686-72be-4b87-b779-87c62131c59b","Type":"ContainerStarted","Data":"0cd8106f7534f1d8933402ba153b52d12bc3d63b40c292005cec4b89d892f87e"}
Sep 11 01:38:01.004351 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:01.004198    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w" event={"ID":"5a2fa686-72be-4b87-b779-87c62131c59b","Type":"ContainerStarted","Data":"2a6dd2f60516c113b72e1d2751b8514a1b92e529e31dc75c34d86f787e5de4ff"}
Sep 11 01:38:01.018896 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:01.018807    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w" podStartSLOduration=1.018786588 podStartE2EDuration="1.018786588s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:38:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:38:01.018260421 +0000 UTC m=+2396.415452594" watchObservedRunningTime="2026-09-11 01:38:01.018786588 +0000 UTC m=+2396.415978763"
Sep 11 01:38:05.791469 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:05.791438    2574 scope.go:117] "RemoveContainer" containerID="fbd7561a281a9141f7e47f1fa5d820c779a8bc7cdd7d3160c4ac3d27c449f7fc"
Sep 11 01:38:05.799336 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:05.799316    2574 scope.go:117] "RemoveContainer" containerID="ea50b35ac07009106f1e00ed116ab5386c3093f3906e387c7631eb220e139229"
Sep 11 01:38:08.785674 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:08.785636    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:38:10.597672 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:10.597637    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w"
Sep 11 01:38:10.598196 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:10.597682    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w"
Sep 11 01:38:10.610211 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:10.610186    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w"
Sep 11 01:38:11.060701 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:11.060670    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w"
Sep 11 01:38:30.467088 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:30.466999    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w"]
Sep 11 01:38:30.487492 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:30.487461    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:38:31.912115 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:31.912069    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:38:45.488097 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:45.488049    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w" podUID="5a2fa686-72be-4b87-b779-87c62131c59b" containerName="main" containerID="cri-o://0cd8106f7534f1d8933402ba153b52d12bc3d63b40c292005cec4b89d892f87e" gracePeriod=45
Sep 11 01:38:45.735979 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:45.735952    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w"
Sep 11 01:38:45.744262 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:45.744198    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a2fa686-72be-4b87-b779-87c62131c59b-home\") pod \"5a2fa686-72be-4b87-b779-87c62131c59b\" (UID: \"5a2fa686-72be-4b87-b779-87c62131c59b\") "
Sep 11 01:38:45.744262 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:45.744235    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a2fa686-72be-4b87-b779-87c62131c59b-tls-certs\") pod \"5a2fa686-72be-4b87-b779-87c62131c59b\" (UID: \"5a2fa686-72be-4b87-b779-87c62131c59b\") "
Sep 11 01:38:45.744449 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:45.744280    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a2fa686-72be-4b87-b779-87c62131c59b-tmp-dir\") pod \"5a2fa686-72be-4b87-b779-87c62131c59b\" (UID: \"5a2fa686-72be-4b87-b779-87c62131c59b\") "
Sep 11 01:38:45.744449 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:45.744295    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a2fa686-72be-4b87-b779-87c62131c59b-model-cache\") pod \"5a2fa686-72be-4b87-b779-87c62131c59b\" (UID: \"5a2fa686-72be-4b87-b779-87c62131c59b\") "
Sep 11 01:38:45.744449 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:45.744319    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85brc\" (UniqueName: \"kubernetes.io/projected/5a2fa686-72be-4b87-b779-87c62131c59b-kube-api-access-85brc\") pod \"5a2fa686-72be-4b87-b779-87c62131c59b\" (UID: \"5a2fa686-72be-4b87-b779-87c62131c59b\") "
Sep 11 01:38:45.744449 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:45.744363    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a2fa686-72be-4b87-b779-87c62131c59b-dshm\") pod \"5a2fa686-72be-4b87-b779-87c62131c59b\" (UID: \"5a2fa686-72be-4b87-b779-87c62131c59b\") "
Sep 11 01:38:45.744643 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:45.744483    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a2fa686-72be-4b87-b779-87c62131c59b-home" (OuterVolumeSpecName: "home") pod "5a2fa686-72be-4b87-b779-87c62131c59b" (UID: "5a2fa686-72be-4b87-b779-87c62131c59b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:38:45.744643 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:45.744536    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a2fa686-72be-4b87-b779-87c62131c59b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5a2fa686-72be-4b87-b779-87c62131c59b" (UID: "5a2fa686-72be-4b87-b779-87c62131c59b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:38:45.744643 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:45.744563    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a2fa686-72be-4b87-b779-87c62131c59b-model-cache" (OuterVolumeSpecName: "model-cache") pod "5a2fa686-72be-4b87-b779-87c62131c59b" (UID: "5a2fa686-72be-4b87-b779-87c62131c59b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:38:45.744643 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:45.744600    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a2fa686-72be-4b87-b779-87c62131c59b-home\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:38:45.744643 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:45.744616    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a2fa686-72be-4b87-b779-87c62131c59b-tmp-dir\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:38:45.746677 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:45.746650    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a2fa686-72be-4b87-b779-87c62131c59b-kube-api-access-85brc" (OuterVolumeSpecName: "kube-api-access-85brc") pod "5a2fa686-72be-4b87-b779-87c62131c59b" (UID: "5a2fa686-72be-4b87-b779-87c62131c59b"). InnerVolumeSpecName "kube-api-access-85brc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:38:45.746789 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:45.746699    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a2fa686-72be-4b87-b779-87c62131c59b-dshm" (OuterVolumeSpecName: "dshm") pod "5a2fa686-72be-4b87-b779-87c62131c59b" (UID: "5a2fa686-72be-4b87-b779-87c62131c59b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:38:45.746789 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:45.746706    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a2fa686-72be-4b87-b779-87c62131c59b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5a2fa686-72be-4b87-b779-87c62131c59b" (UID: "5a2fa686-72be-4b87-b779-87c62131c59b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:38:45.845105 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:45.845081    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a2fa686-72be-4b87-b779-87c62131c59b-dshm\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:38:45.845105 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:45.845104    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a2fa686-72be-4b87-b779-87c62131c59b-tls-certs\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:38:45.845247 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:45.845114    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a2fa686-72be-4b87-b779-87c62131c59b-model-cache\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:38:45.845247 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:45.845122    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85brc\" (UniqueName: \"kubernetes.io/projected/5a2fa686-72be-4b87-b779-87c62131c59b-kube-api-access-85brc\") on node \"ip-10-0-139-235.ec2.internal\" DevicePath \"\""
Sep 11 01:38:46.207360 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:46.207327    2574 generic.go:356] "Generic (PLEG): container finished" podID="5a2fa686-72be-4b87-b779-87c62131c59b" containerID="0cd8106f7534f1d8933402ba153b52d12bc3d63b40c292005cec4b89d892f87e" exitCode=0
Sep 11 01:38:46.207508 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:46.207365    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w"
Sep 11 01:38:46.207508 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:46.207396    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w" event={"ID":"5a2fa686-72be-4b87-b779-87c62131c59b","Type":"ContainerDied","Data":"0cd8106f7534f1d8933402ba153b52d12bc3d63b40c292005cec4b89d892f87e"}
Sep 11 01:38:46.207508 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:46.207431    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w" event={"ID":"5a2fa686-72be-4b87-b779-87c62131c59b","Type":"ContainerDied","Data":"2a6dd2f60516c113b72e1d2751b8514a1b92e529e31dc75c34d86f787e5de4ff"}
Sep 11 01:38:46.207508 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:46.207448    2574 scope.go:117] "RemoveContainer" containerID="0cd8106f7534f1d8933402ba153b52d12bc3d63b40c292005cec4b89d892f87e"
Sep 11 01:38:46.216514 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:46.216492    2574 scope.go:117] "RemoveContainer" containerID="0cd8106f7534f1d8933402ba153b52d12bc3d63b40c292005cec4b89d892f87e"
Sep 11 01:38:46.216754 ip-10-0-139-235 kubenswrapper[2574]: E0911 01:38:46.216737    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cd8106f7534f1d8933402ba153b52d12bc3d63b40c292005cec4b89d892f87e\": container with ID starting with 0cd8106f7534f1d8933402ba153b52d12bc3d63b40c292005cec4b89d892f87e not found: ID does not exist" containerID="0cd8106f7534f1d8933402ba153b52d12bc3d63b40c292005cec4b89d892f87e"
Sep 11 01:38:46.216808 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:46.216763    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cd8106f7534f1d8933402ba153b52d12bc3d63b40c292005cec4b89d892f87e"} err="failed to get container status \"0cd8106f7534f1d8933402ba153b52d12bc3d63b40c292005cec4b89d892f87e\": rpc error: code = NotFound desc = could not find container \"0cd8106f7534f1d8933402ba153b52d12bc3d63b40c292005cec4b89d892f87e\": container with ID starting with 0cd8106f7534f1d8933402ba153b52d12bc3d63b40c292005cec4b89d892f87e not found: ID does not exist"
Sep 11 01:38:46.226610 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:46.226589    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w"]
Sep 11 01:38:46.232101 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:46.232082    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-d3e6934d-kserve-7b4cfd997f-q6d5w"]
Sep 11 01:38:47.100805 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:47.100770    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a2fa686-72be-4b87-b779-87c62131c59b" path="/var/lib/kubelet/pods/5a2fa686-72be-4b87-b779-87c62131c59b/volumes"
Sep 11 01:38:51.946717 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:38:51.946655    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:39:05.819278 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:05.819157    2574 scope.go:117] "RemoveContainer" containerID="5971acf7bd8f958a78169b8720cd9ec753ac8d351da6a1cb2012632e87a7f3be"
Sep 11 01:39:21.434442 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:21.434369    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6db4d697d8-ljjlp_71578834-02d3-4bc8-ba21-c911a82c1ab0/authorino/0.log"
Sep 11 01:39:27.853735 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:27.853703    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8bd47c7d5-nvrtp_86bf8c2c-cd19-484f-8b03-659d42056d67/copy-manifests/0.log"
Sep 11 01:39:27.860572 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:27.860546    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8bd47c7d5-nvrtp_86bf8c2c-cd19-484f-8b03-659d42056d67/manager/0.log"
Sep 11 01:39:28.677458 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:28.677425    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29818170-wl98m_bbab7a05-c5ce-4039-a867-24421fd51d48/cleanup/0.log"
Sep 11 01:39:29.618942 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:29.618868    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6db4d697d8-ljjlp_71578834-02d3-4bc8-ba21-c911a82c1ab0/authorino/0.log"
Sep 11 01:39:29.856733 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:29.856702    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-ws2mz_cb5c9843-cc67-456f-a22d-fa23447c7f4b/manager/0.log"
Sep 11 01:39:29.970505 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:29.970475    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-jmkt5_42094776-f11a-4606-a02a-8ab6829214bf/kuadrant-console-plugin/0.log"
Sep 11 01:39:30.171501 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:30.171468    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:39:30.396593 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:30.396502    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-rdhcs_954c463d-8df1-4d71-a781-ddbb9f1c1282/manager/0.log"
Sep 11 01:39:31.102901 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:31.102868    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58fc6d6988-dvf8m_8483ab9d-a863-4d4b-93bd-8ca7d451c493/kube-auth-proxy/0.log"
Sep 11 01:39:31.354033 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:31.353952    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-dxdkg_ef51a3ee-99ca-4c0f-bf70-1bf2b8791cc5/payload-pre-processing/0.log"
Sep 11 01:39:32.062339 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:32.062306    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-gk5b8_eb818e57-a9c5-41ac-b057-df2fb48e5b86/main/0.log"
Sep 11 01:39:32.178478 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:32.178440    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-zq88v_f72f5638-fbb0-423a-8620-994a1529bf99/main/0.log"
Sep 11 01:39:38.084454 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:38.084415    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sjvqr/must-gather-wddvr"]
Sep 11 01:39:38.084974 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:38.084771    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a2fa686-72be-4b87-b779-87c62131c59b" containerName="main"
Sep 11 01:39:38.084974 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:38.084782    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a2fa686-72be-4b87-b779-87c62131c59b" containerName="main"
Sep 11 01:39:38.084974 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:38.084835    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a2fa686-72be-4b87-b779-87c62131c59b" containerName="main"
Sep 11 01:39:38.088016 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:38.087998    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sjvqr/must-gather-wddvr"
Sep 11 01:39:38.090307 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:38.090287    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-sjvqr\"/\"default-dockercfg-dv4qx\""
Sep 11 01:39:38.090708 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:38.090309    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sjvqr\"/\"openshift-service-ca.crt\""
Sep 11 01:39:38.090708 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:38.090604    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sjvqr\"/\"kube-root-ca.crt\""
Sep 11 01:39:38.097331 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:38.097307    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:39:38.097512 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:38.097483    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sjvqr/must-gather-wddvr"]
Sep 11 01:39:38.186625 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:38.186597    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4f3f2db9-82e2-4956-b545-812fee4f9ae8-must-gather-output\") pod \"must-gather-wddvr\" (UID: \"4f3f2db9-82e2-4956-b545-812fee4f9ae8\") " pod="openshift-must-gather-sjvqr/must-gather-wddvr"
Sep 11 01:39:38.186784 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:38.186634    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wkgt\" (UniqueName: \"kubernetes.io/projected/4f3f2db9-82e2-4956-b545-812fee4f9ae8-kube-api-access-7wkgt\") pod \"must-gather-wddvr\" (UID: \"4f3f2db9-82e2-4956-b545-812fee4f9ae8\") " pod="openshift-must-gather-sjvqr/must-gather-wddvr"
Sep 11 01:39:38.192352 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:38.192322    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:39:38.287766 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:38.287734    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4f3f2db9-82e2-4956-b545-812fee4f9ae8-must-gather-output\") pod \"must-gather-wddvr\" (UID: \"4f3f2db9-82e2-4956-b545-812fee4f9ae8\") " pod="openshift-must-gather-sjvqr/must-gather-wddvr"
Sep 11 01:39:38.287953 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:38.287771    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wkgt\" (UniqueName: \"kubernetes.io/projected/4f3f2db9-82e2-4956-b545-812fee4f9ae8-kube-api-access-7wkgt\") pod \"must-gather-wddvr\" (UID: \"4f3f2db9-82e2-4956-b545-812fee4f9ae8\") " pod="openshift-must-gather-sjvqr/must-gather-wddvr"
Sep 11 01:39:38.288162 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:38.288144    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4f3f2db9-82e2-4956-b545-812fee4f9ae8-must-gather-output\") pod \"must-gather-wddvr\" (UID: \"4f3f2db9-82e2-4956-b545-812fee4f9ae8\") " pod="openshift-must-gather-sjvqr/must-gather-wddvr"
Sep 11 01:39:38.295700 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:38.295673    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wkgt\" (UniqueName: \"kubernetes.io/projected/4f3f2db9-82e2-4956-b545-812fee4f9ae8-kube-api-access-7wkgt\") pod \"must-gather-wddvr\" (UID: \"4f3f2db9-82e2-4956-b545-812fee4f9ae8\") " pod="openshift-must-gather-sjvqr/must-gather-wddvr"
Sep 11 01:39:38.398839 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:38.398757    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sjvqr/must-gather-wddvr"
Sep 11 01:39:38.667958 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:38.667925    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sjvqr/must-gather-wddvr"]
Sep 11 01:39:38.670761 ip-10-0-139-235 kubenswrapper[2574]: W0911 01:39:38.670734    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f3f2db9_82e2_4956_b545_812fee4f9ae8.slice/crio-bdef88ae78b479e00bd7d1c8f97d45bbd9c70e79ee281b07671eac056dd863a5 WatchSource:0}: Error finding container bdef88ae78b479e00bd7d1c8f97d45bbd9c70e79ee281b07671eac056dd863a5: Status 404 returned error can't find the container with id bdef88ae78b479e00bd7d1c8f97d45bbd9c70e79ee281b07671eac056dd863a5
Sep 11 01:39:38.672815 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:38.672795    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 01:39:39.458265 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:39.458092    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sjvqr/must-gather-wddvr" event={"ID":"4f3f2db9-82e2-4956-b545-812fee4f9ae8","Type":"ContainerStarted","Data":"579a1e985c49a2f4112d3e5e79da54fbe3ba7788bc87dcaaed8133c0f0ed6993"}
Sep 11 01:39:39.458265 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:39.458145    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sjvqr/must-gather-wddvr" event={"ID":"4f3f2db9-82e2-4956-b545-812fee4f9ae8","Type":"ContainerStarted","Data":"bdef88ae78b479e00bd7d1c8f97d45bbd9c70e79ee281b07671eac056dd863a5"}
Sep 11 01:39:40.467666 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:40.467608    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sjvqr/must-gather-wddvr" event={"ID":"4f3f2db9-82e2-4956-b545-812fee4f9ae8","Type":"ContainerStarted","Data":"6a1fa2e56e89605b8840101272d7a013d168e56c1aafcd97bfa1adf6de8a831a"}
Sep 11 01:39:40.486168 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:40.486113    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-sjvqr/must-gather-wddvr" podStartSLOduration=1.8178088209999999 podStartE2EDuration="2.486095714s" totalImagesPullingTime="668.286893ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:39:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:39:40.484227109 +0000 UTC m=+2495.881419283" watchObservedRunningTime="2026-09-11 01:39:40.486095714 +0000 UTC m=+2495.883287932"
Sep 11 01:39:40.949846 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:40.949808    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-jbpsj_ff3ee2a6-e101-4cde-8d89-489f1db81ab8/global-pull-secret-syncer/0.log"
Sep 11 01:39:41.034847 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:41.034812    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log"
Sep 11 01:39:41.145679 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:41.145650    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-235.ec2.internal_9f1f5adc45e045b22e89459332dfb6a0/haproxy/0.log"
Sep 11 01:39:44.555714 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:44.555655    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:39:44.637864 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:44.637814    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l5nhz_396a7cf9-e334-407d-ad45-789fb8dfba05/konnectivity-agent/0.log" limit=1024
Sep 11 01:39:45.748410 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:45.748147    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6db4d697d8-ljjlp_71578834-02d3-4bc8-ba21-c911a82c1ab0/authorino/0.log"
Sep 11 01:39:45.806751 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:45.806713    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-ws2mz_cb5c9843-cc67-456f-a22d-fa23447c7f4b/manager/0.log"
Sep 11 01:39:45.832959 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:45.832920    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-jmkt5_42094776-f11a-4606-a02a-8ab6829214bf/kuadrant-console-plugin/0.log"
Sep 11 01:39:45.989935 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:45.987825    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-rdhcs_954c463d-8df1-4d71-a781-ddbb9f1c1282/manager/0.log"
Sep 11 01:39:48.008471 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:48.008384    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-564878c586-bhl5l_2b808fe3-eb44-4658-bb1e-524852e8fb66/metrics-server/0.log"
Sep 11 01:39:48.077201 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:48.075851    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9rdkv_9b1169c4-a910-4cb0-85f4-dcccb089a5cb/node-exporter/0.log"
Sep 11 01:39:48.109857 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:48.109779    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9rdkv_9b1169c4-a910-4cb0-85f4-dcccb089a5cb/kube-rbac-proxy/0.log"
Sep 11 01:39:48.137210 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:48.137178    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9rdkv_9b1169c4-a910-4cb0-85f4-dcccb089a5cb/init-textfile/0.log"
Sep 11 01:39:48.358018 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:48.357901    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-5nbw5_0aa988f8-e121-432c-8be1-90595aeb64a0/kube-rbac-proxy-main/0.log"
Sep 11 01:39:48.382224 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:48.382192    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-5nbw5_0aa988f8-e121-432c-8be1-90595aeb64a0/kube-rbac-proxy-self/0.log"
Sep 11 01:39:48.404343 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:48.404312    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-5nbw5_0aa988f8-e121-432c-8be1-90595aeb64a0/openshift-state-metrics/0.log"
Sep 11 01:39:48.450708 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:48.450659    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e83f749b-c698-4e1b-a415-5a532db7da0b/prometheus/0.log"
Sep 11 01:39:48.470339 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:48.470303    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e83f749b-c698-4e1b-a415-5a532db7da0b/config-reloader/0.log"
Sep 11 01:39:48.492074 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:48.492046    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e83f749b-c698-4e1b-a415-5a532db7da0b/thanos-sidecar/0.log"
Sep 11 01:39:48.515951 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:48.515902    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e83f749b-c698-4e1b-a415-5a532db7da0b/kube-rbac-proxy-web/0.log"
Sep 11 01:39:48.539216 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:48.539181    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e83f749b-c698-4e1b-a415-5a532db7da0b/kube-rbac-proxy/0.log"
Sep 11 01:39:48.566536 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:48.566506    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e83f749b-c698-4e1b-a415-5a532db7da0b/kube-rbac-proxy-thanos/0.log"
Sep 11 01:39:48.591932 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:48.591032    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e83f749b-c698-4e1b-a415-5a532db7da0b/init-config-reloader/0.log"
Sep 11 01:39:51.493840 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:51.493797    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sjvqr/perf-node-gather-daemonset-pmdcf"]
Sep 11 01:39:51.501879 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:51.501843    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-pmdcf"
Sep 11 01:39:51.505401 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:51.505363    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sjvqr/perf-node-gather-daemonset-pmdcf"]
Sep 11 01:39:51.649188 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:51.649148    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c5fad79b-b5bd-4e62-b630-f03acf896fca-proc\") pod \"perf-node-gather-daemonset-pmdcf\" (UID: \"c5fad79b-b5bd-4e62-b630-f03acf896fca\") " pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-pmdcf"
Sep 11 01:39:51.649395 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:51.649199    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smt2g\" (UniqueName: \"kubernetes.io/projected/c5fad79b-b5bd-4e62-b630-f03acf896fca-kube-api-access-smt2g\") pod \"perf-node-gather-daemonset-pmdcf\" (UID: \"c5fad79b-b5bd-4e62-b630-f03acf896fca\") " pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-pmdcf"
Sep 11 01:39:51.649395 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:51.649234    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c5fad79b-b5bd-4e62-b630-f03acf896fca-podres\") pod \"perf-node-gather-daemonset-pmdcf\" (UID: \"c5fad79b-b5bd-4e62-b630-f03acf896fca\") " pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-pmdcf"
Sep 11 01:39:51.649395 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:51.649268    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c5fad79b-b5bd-4e62-b630-f03acf896fca-lib-modules\") pod \"perf-node-gather-daemonset-pmdcf\" (UID: \"c5fad79b-b5bd-4e62-b630-f03acf896fca\") " pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-pmdcf"
Sep 11 01:39:51.649395 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:51.649349    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c5fad79b-b5bd-4e62-b630-f03acf896fca-sys\") pod \"perf-node-gather-daemonset-pmdcf\" (UID: \"c5fad79b-b5bd-4e62-b630-f03acf896fca\") " pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-pmdcf"
Sep 11 01:39:51.750617 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:51.750509    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c5fad79b-b5bd-4e62-b630-f03acf896fca-podres\") pod \"perf-node-gather-daemonset-pmdcf\" (UID: \"c5fad79b-b5bd-4e62-b630-f03acf896fca\") " pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-pmdcf"
Sep 11 01:39:51.750791 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:51.750621    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c5fad79b-b5bd-4e62-b630-f03acf896fca-lib-modules\") pod \"perf-node-gather-daemonset-pmdcf\" (UID: \"c5fad79b-b5bd-4e62-b630-f03acf896fca\") " pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-pmdcf"
Sep 11 01:39:51.750791 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:51.750725    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c5fad79b-b5bd-4e62-b630-f03acf896fca-sys\") pod \"perf-node-gather-daemonset-pmdcf\" (UID: \"c5fad79b-b5bd-4e62-b630-f03acf896fca\") " pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-pmdcf"
Sep 11 01:39:51.750943 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:51.750806    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c5fad79b-b5bd-4e62-b630-f03acf896fca-lib-modules\") pod \"perf-node-gather-daemonset-pmdcf\" (UID: \"c5fad79b-b5bd-4e62-b630-f03acf896fca\") " pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-pmdcf"
Sep 11 01:39:51.750943 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:51.750832    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c5fad79b-b5bd-4e62-b630-f03acf896fca-sys\") pod \"perf-node-gather-daemonset-pmdcf\" (UID: \"c5fad79b-b5bd-4e62-b630-f03acf896fca\") " pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-pmdcf"
Sep 11 01:39:51.750943 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:51.750938    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c5fad79b-b5bd-4e62-b630-f03acf896fca-podres\") pod \"perf-node-gather-daemonset-pmdcf\" (UID: \"c5fad79b-b5bd-4e62-b630-f03acf896fca\") " pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-pmdcf"
Sep 11 01:39:51.751107 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:51.750958    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c5fad79b-b5bd-4e62-b630-f03acf896fca-proc\") pod \"perf-node-gather-daemonset-pmdcf\" (UID: \"c5fad79b-b5bd-4e62-b630-f03acf896fca\") " pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-pmdcf"
Sep 11 01:39:51.751107 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:51.750990    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smt2g\" (UniqueName: \"kubernetes.io/projected/c5fad79b-b5bd-4e62-b630-f03acf896fca-kube-api-access-smt2g\") pod \"perf-node-gather-daemonset-pmdcf\" (UID: \"c5fad79b-b5bd-4e62-b630-f03acf896fca\") " pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-pmdcf"
Sep 11 01:39:51.751107 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:51.751073    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c5fad79b-b5bd-4e62-b630-f03acf896fca-proc\") pod \"perf-node-gather-daemonset-pmdcf\" (UID: \"c5fad79b-b5bd-4e62-b630-f03acf896fca\") " pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-pmdcf"
Sep 11 01:39:51.758966 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:51.758936    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smt2g\" (UniqueName: \"kubernetes.io/projected/c5fad79b-b5bd-4e62-b630-f03acf896fca-kube-api-access-smt2g\") pod \"perf-node-gather-daemonset-pmdcf\" (UID: \"c5fad79b-b5bd-4e62-b630-f03acf896fca\") " pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-pmdcf"
Sep 11 01:39:51.818656 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:51.818618    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-pmdcf"
Sep 11 01:39:52.109697 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:52.109661    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sjvqr/perf-node-gather-daemonset-pmdcf"]
Sep 11 01:39:52.590719 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:52.590672    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-pmdcf" event={"ID":"c5fad79b-b5bd-4e62-b630-f03acf896fca","Type":"ContainerStarted","Data":"1c8360c150c20defb2f1107f982dd821b90ccefa3582325c7f335b4449fc561b"}
Sep 11 01:39:52.591237 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:52.590774    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-pmdcf" event={"ID":"c5fad79b-b5bd-4e62-b630-f03acf896fca","Type":"ContainerStarted","Data":"8871c222e8a7e5354d07054152643add941fb1c736bfc1b0b0de566597a8e224"}
Sep 11 01:39:52.591237 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:52.590820    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-pmdcf"
Sep 11 01:39:52.606595 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:52.606528    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-pmdcf" podStartSLOduration=1.606503321 podStartE2EDuration="1.606503321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 01:39:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:39:52.605254995 +0000 UTC m=+2508.002447170" watchObservedRunningTime="2026-09-11 01:39:52.606503321 +0000 UTC m=+2508.003695496"
Sep 11 01:39:53.337548 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:53.337521    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-j9mpz_dda2c7fa-a50c-483c-8709-775d04d924c2/dns/0.log"
Sep 11 01:39:53.357462 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:53.357433    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-j9mpz_dda2c7fa-a50c-483c-8709-775d04d924c2/kube-rbac-proxy/0.log"
Sep 11 01:39:53.510323 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:53.510284    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zw22q_d097ee33-8900-43b7-9eae-35ac88373d1d/dns-node-resolver/0.log"
Sep 11 01:39:54.085783 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:54.085755    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-84ccd59f5-rz4qk_edba57c7-d190-4a32-8614-2b18ed243970/registry/0.log"
Sep 11 01:39:54.131555 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:54.131492    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hh5wm_1625cd1f-5711-4d05-9cfc-74a2a2c1c637/node-ca/0.log"
Sep 11 01:39:55.585082 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:55.585053    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58fc6d6988-dvf8m_8483ab9d-a863-4d4b-93bd-8ca7d451c493/kube-auth-proxy/0.log"
Sep 11 01:39:55.662149 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:55.662114    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-dxdkg_ef51a3ee-99ca-4c0f-bf70-1bf2b8791cc5/payload-pre-processing/0.log"
Sep 11 01:39:56.600227 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:56.600130    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-cs2xq_af240b86-c806-4cf9-a3f0-0bee21e7abfe/serve-healthcheck-canary/0.log"
Sep 11 01:39:57.222129 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:57.222102    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v9q8v_000696c7-5f3f-469b-a8bd-3df2d07590b4/kube-rbac-proxy/0.log"
Sep 11 01:39:57.242032 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:57.241997    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v9q8v_000696c7-5f3f-469b-a8bd-3df2d07590b4/exporter/0.log"
Sep 11 01:39:57.261899 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:57.261865    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v9q8v_000696c7-5f3f-469b-a8bd-3df2d07590b4/extractor/0.log"
Sep 11 01:39:58.608840 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:58.608804    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-pmdcf"
Sep 11 01:39:59.814585 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:59.814555    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8bd47c7d5-nvrtp_86bf8c2c-cd19-484f-8b03-659d42056d67/manager/0.log"
Sep 11 01:39:59.838638 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:39:59.838608    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8bd47c7d5-nvrtp_86bf8c2c-cd19-484f-8b03-659d42056d67/copy-manifests/0.log"
Sep 11 01:40:01.336550 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:40:01.336517    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5bdf494464-dk7ld_2e80d19c-5d88-4e0c-9a78-a45dfa9b3c7e/manager/0.log"
Sep 11 01:40:01.392470 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:40:01.392391    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-rssq5_57ea4d22-0d1f-49b4-84f4-dab05cc44f84/openshift-lws-operator/0.log"
Sep 11 01:40:02.591924 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:40:02.591870    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58fc6d6988-dvf8m_8483ab9d-a863-4d4b-93bd-8ca7d451c493/kube-auth-proxy/0.log"
Sep 11 01:40:02.668451 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:40:02.668417    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-dxdkg_ef51a3ee-99ca-4c0f-bf70-1bf2b8791cc5/payload-pre-processing/0.log"
Sep 11 01:40:04.947664 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:40:04.947631    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58fc6d6988-dvf8m_8483ab9d-a863-4d4b-93bd-8ca7d451c493/kube-auth-proxy/0.log"
Sep 11 01:40:05.008311 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:40:05.008266    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-dxdkg_ef51a3ee-99ca-4c0f-bf70-1bf2b8791cc5/payload-pre-processing/0.log"
Sep 11 01:40:05.841970 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:40:05.841938    2574 scope.go:117] "RemoveContainer" containerID="6344b5899888a15530084ac6d6edf56ecf6b8ce13aec6bc8b1cabcd21a99a40f"
Sep 11 01:40:06.393139 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:40:06.393100    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-hj6rz_8fd16e7e-9fe7-499d-becf-de3ba31c8549/migrator/0.log"
Sep 11 01:40:06.411382 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:40:06.411357    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-hj6rz_8fd16e7e-9fe7-499d-becf-de3ba31c8549/graceful-termination/0.log"
Sep 11 01:40:08.005328 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:40:08.005304    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7j9mn_1fe146d4-3a94-48d2-b383-ea0a4c301b54/kube-multus-additional-cni-plugins/0.log"
Sep 11 01:40:08.024251 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:40:08.024219    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7j9mn_1fe146d4-3a94-48d2-b383-ea0a4c301b54/egress-router-binary-copy/0.log"
Sep 11 01:40:08.044970 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:40:08.044941    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7j9mn_1fe146d4-3a94-48d2-b383-ea0a4c301b54/cni-plugins/0.log"
Sep 11 01:40:08.065166 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:40:08.065136    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7j9mn_1fe146d4-3a94-48d2-b383-ea0a4c301b54/bond-cni-plugin/0.log"
Sep 11 01:40:08.083928 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:40:08.083882    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7j9mn_1fe146d4-3a94-48d2-b383-ea0a4c301b54/routeoverride-cni/0.log"
Sep 11 01:40:08.110164 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:40:08.110137    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7j9mn_1fe146d4-3a94-48d2-b383-ea0a4c301b54/whereabouts-cni-bincopy/0.log"
Sep 11 01:40:08.130410 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:40:08.130382    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7j9mn_1fe146d4-3a94-48d2-b383-ea0a4c301b54/whereabouts-cni/0.log"
Sep 11 01:40:08.430670 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:40:08.430638    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-w9q8q_f4363dbb-d8eb-4eb6-af98-f3327b90c84c/kube-multus/0.log"
Sep 11 01:40:08.495398 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:40:08.495359    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9s5p9_77093864-53b6-4044-952a-d8f1a965dd1d/network-metrics-daemon/0.log"
Sep 11 01:40:08.509618 ip-10-0-139-235 kubenswrapper[2574]: I0911 01:40:08.509591    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9s5p9_77093864-53b6-4044-952a-d8f1a965dd1d/kube-rbac-proxy/0.log"